[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:43:39 dist-test-slave-k5rr krb5kdc[26167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:43:39 dist-test-slave-k5rr krb5kdc[26167](info): set up 2 sockets
Aug 08 02:43:39 dist-test-slave-k5rr krb5kdc[26167](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 02:43:40.617983 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.006s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:43:40 dist-test-slave-k5rr krb5kdc[26167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:43:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:43:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:43:40.811448 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:39519
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:39983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:39519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:41.287057 26186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:41.287498 26186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:41.287667 26186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:41.295382 26186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:43:41.295504 26186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:41.295591 26186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:41.295672 26186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:43:41.295751 26186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:43:41.306263 26186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:39983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:39519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:39519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:41.308187 26186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:41.310223 26186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:41.332890 26195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:41.332917 26192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:41.333030 26197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:41.336349 26186 server_base.cc:1034] running on GCE node
I20260808 02:43:41.337633 26186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:41.339985 26186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:41.341270 26186 hybrid_clock.cc:648] HybridClock initialized: now 1786157021341199 us; error 95 us; skew 500 ppm
I20260808 02:43:41.341917 26186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:41 dist-test-slave-k5rr krb5kdc[26167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157021, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:41.345799 26186 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:43:41.348495 26186 webserver.cc:460] Webserver started at http://127.25.137.254:45681/ using document root <none> and password file <none>
I20260808 02:43:41.349689 26186 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:41.349828 26186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:41.350317 26186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:41.354777 26186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "4d9229e02c674d9bb5362c1cfc05ed75"
format_stamp: "Formatted at 2026-08-08 02:43:41 on dist-test-slave-k5rr"
server_key: "21226deb0ae9771ab7c12a8e73777a79"
server_key_iv: "ea1e58075798322708861f88ecd8f0f8"
server_key_version: "encryptionkey@0"
I20260808 02:43:41.355759 26186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "4d9229e02c674d9bb5362c1cfc05ed75"
format_stamp: "Formatted at 2026-08-08 02:43:41 on dist-test-slave-k5rr"
server_key: "21226deb0ae9771ab7c12a8e73777a79"
server_key_iv: "ea1e58075798322708861f88ecd8f0f8"
server_key_version: "encryptionkey@0"
I20260808 02:43:41.363832 26186 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260808 02:43:41.369766 26214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:41.371838 26186 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260808 02:43:41.372123 26186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "4d9229e02c674d9bb5362c1cfc05ed75"
format_stamp: "Formatted at 2026-08-08 02:43:41 on dist-test-slave-k5rr"
server_key: "21226deb0ae9771ab7c12a8e73777a79"
server_key_iv: "ea1e58075798322708861f88ecd8f0f8"
server_key_version: "encryptionkey@0"
I20260808 02:43:41.372349 26186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:41.396569 26186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:41.401846 26186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:41.402280 26186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:41.424134 26186 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:39519
I20260808 02:43:41.424163 26289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:39519 every 8 connection(s)
I20260808 02:43:41.426505 26186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:43:41.433588 26292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:41.434759 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 26186
I20260808 02:43:41.435180 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:43:41.438222 26151 external_mini_cluster.cc:1428] Setting key 0b0847c120c35d309deb00a4595d5053
I20260808 02:43:41.454174 26292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75: Bootstrap starting.
I20260808 02:43:41.461575 26292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:41.463709 26292 log.cc:824] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:41.468273 26292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75: No bootstrap required, opened a new log
Aug 08 02:43:41 dist-test-slave-k5rr krb5kdc[26167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:41.480996 26292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9229e02c674d9bb5362c1cfc05ed75" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39519 } }
I20260808 02:43:41.481766 26292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:41.481977 26292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9229e02c674d9bb5362c1cfc05ed75, State: Initialized, Role: FOLLOWER
I20260808 02:43:41.483350 26292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9229e02c674d9bb5362c1cfc05ed75" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39519 } }
I20260808 02:43:41.483811 26292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:43:41.484135 26292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:43:41.484408 26292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:41.487730 26292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9229e02c674d9bb5362c1cfc05ed75" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39519 } }
I20260808 02:43:41.488920 26292 leader_election.cc:302] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d9229e02c674d9bb5362c1cfc05ed75; no voters:
I20260808 02:43:41.490406 26292 leader_election.cc:288] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:43:41.490593 26301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:43:41.491586 26299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:41.447791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46364 (local address 127.25.137.254:39519)
0808 02:43:41.448702 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:41.448713 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:41.448761 (+ 48us) server_negotiation.cc:407] Connection header received
0808 02:43:41.460932 (+ 12171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:41.460955 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:41.461602 (+ 647us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:41.462013 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:41.466547 (+ 4534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:41.468239 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:41.470977 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:41.471797 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:41.472023 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:41.479787 (+ 7764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:41.479854 (+ 67us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:41.479906 (+ 52us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:41.479957 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:41.484862 (+ 4905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:41.485824 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:41.485840 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:41.485849 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:41.486127 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:41.486625 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:41.486631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:41.486635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:41.487559 (+ 924us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:41.488009 (+ 450us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:41.488975 (+ 966us) server_negotiation.cc:299] Negotiation successful
0808 02:43:41.489493 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":386,"threads_started":1}
I20260808 02:43:41.493337 26301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [term 1 LEADER]: Becoming Leader. State: Replica: 4d9229e02c674d9bb5362c1cfc05ed75, State: Running, Role: LEADER
I20260808 02:43:41.493678 26292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:43:41.494313 26301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9229e02c674d9bb5362c1cfc05ed75" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39519 } }
I20260808 02:43:41.501294 26305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d9229e02c674d9bb5362c1cfc05ed75. Latest consensus state: current_term: 1 leader_uuid: "4d9229e02c674d9bb5362c1cfc05ed75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9229e02c674d9bb5362c1cfc05ed75" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39519 } } }
I20260808 02:43:41.501756 26305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [sys.catalog]: This master's current role is: LEADER
I20260808 02:43:41.501502 26304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d9229e02c674d9bb5362c1cfc05ed75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9229e02c674d9bb5362c1cfc05ed75" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39519 } } }
I20260808 02:43:41.501909 26304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75 [sys.catalog]: This master's current role is: LEADER
W20260808 02:43:41.509740 26328 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:43:41.509909 26328 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:43:41.510097 26329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:43:41.512885 26329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:43:41.526726 26329 catalog_manager.cc:1347] Generated new cluster ID: 48af77f372e04a429db2aa1bab85550f
I20260808 02:43:41.526850 26329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:43:41.545270 26329 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:43:41.546825 26329 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:43:41.558792 26329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d9229e02c674d9bb5362c1cfc05ed75: Generated new TSK 0
I20260808 02:43:41.559827 26329 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:43:41.621229 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39519
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:39983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:41.963620 26336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:41.963912 26336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:41.964052 26336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:41.971385 26336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:41.971517 26336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:41.971669 26336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:43:41.981721 26336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:39983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39519
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:41.982983 26336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:41.984290 26336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:41.998380 26347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:41.998402 26350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:41.998364 26348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:41.999150 26336 server_base.cc:1034] running on GCE node
I20260808 02:43:41.999725 26336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:42.000633 26336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:42.001834 26336 hybrid_clock.cc:648] HybridClock initialized: now 1786157022001777 us; error 47 us; skew 500 ppm
I20260808 02:43:42.002127 26336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:41 dist-test-slave-k5rr krb5kdc[26167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157021, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:42.005498 26336 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:43:42.006968 26336 webserver.cc:460] Webserver started at http://127.25.137.193:43803/ using document root <none> and password file <none>
I20260808 02:43:42.007661 26336 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:42.007783 26336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:42.008093 26336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:42.011102 26336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "d2363fd7e8c04d7e98f7c26711c26380"
format_stamp: "Formatted at 2026-08-08 02:43:42 on dist-test-slave-k5rr"
server_key: "c95f4a2399fd0c02794957978f28d4a9"
server_key_iv: "05161a0b0a4851ff0fe366e3f20cb22d"
server_key_version: "encryptionkey@0"
I20260808 02:43:42.011878 26336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "d2363fd7e8c04d7e98f7c26711c26380"
format_stamp: "Formatted at 2026-08-08 02:43:42 on dist-test-slave-k5rr"
server_key: "c95f4a2399fd0c02794957978f28d4a9"
server_key_iv: "05161a0b0a4851ff0fe366e3f20cb22d"
server_key_version: "encryptionkey@0"
I20260808 02:43:42.017076 26336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 02:43:42.020536 26362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:42.022011 26336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 02:43:42.022167 26336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "d2363fd7e8c04d7e98f7c26711c26380"
format_stamp: "Formatted at 2026-08-08 02:43:42 on dist-test-slave-k5rr"
server_key: "c95f4a2399fd0c02794957978f28d4a9"
server_key_iv: "05161a0b0a4851ff0fe366e3f20cb22d"
server_key_version: "encryptionkey@0"
I20260808 02:43:42.022348 26336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:42.033541 26336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:42.037695 26336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:42.037961 26336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:42.039067 26336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:43:42.040783 26336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:42.040858 26336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:42.041069 26336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:42.041141 26336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 02:43:42.073081 26336 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:33873
I20260808 02:43:42.073119 26539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:33873 every 8 connection(s)
I20260808 02:43:42.074378 26336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:43:42.082665 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 26336
I20260808 02:43:42.082921 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:43:42.083722 26151 external_mini_cluster.cc:1428] Setting key e3756009b3d7262853637dbda502fe83
Aug 08 02:43:42 dist-test-slave-k5rr krb5kdc[26167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157021, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:42.097499 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:42.078015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:43331 (local address 127.25.137.254:39519)
0808 02:43:42.078493 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:42.078498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:42.079388 (+ 890us) server_negotiation.cc:407] Connection header received
0808 02:43:42.081593 (+ 2205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:42.081599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:42.081678 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:42.081778 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:42.083858 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:42.084604 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:42.086588 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:42.087003 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:42.087109 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:42.091372 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:42.091378 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:42.091381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:42.091414 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:42.094171 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:42.094986 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:42.094991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:42.094994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:42.095159 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:42.095599 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:42.095604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:42.095607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:42.095998 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:42.096137 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:42.096883 (+ 746us) server_negotiation.cc:299] Negotiation successful
0808 02:43:42.097130 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":249,"threads_started":1}
I20260808 02:43:42.098403 26548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:42.078476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:39519 (local address 127.25.137.193:43331)
0808 02:43:42.079212 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:42.079268 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:42.081321 (+ 2053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:42.082058 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:42.082066 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:42.082825 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:42.083635 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:42.083661 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:42.084767 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:42.084773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:42.086402 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:42.086425 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:42.087246 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:42.087254 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:42.087557 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:42.088305 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:42.088339 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:42.091116 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:42.094326 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:42.094333 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:42.094346 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:42.094862 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:42.095263 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:42.095267 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:42.095269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:42.095476 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:42.096130 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:42.096136 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:42.096455 (+ 319us) client_negotiation.cc:769] Sending connection context
0808 02:43:42.096744 (+ 289us) client_negotiation.cc:240] Negotiation successful
0808 02:43:42.097223 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":276,"threads_started":1}
I20260808 02:43:42.100145 26541 heartbeater.cc:347] Connected to a master server at 127.25.137.254:39519
I20260808 02:43:42.100632 26541 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:42.101508 26541 heartbeater.cc:511] Master 127.25.137.254:39519 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
I20260808 02:43:42.105705 26236 ts_manager.cc:194] Registered new tserver with Master: d2363fd7e8c04d7e98f7c26711c26380 (127.25.137.193:33873)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
I20260808 02:43:42.108042 26236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:43331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:43:42.131455 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39519
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:39983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:42.461464 26553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:42.461784 26553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:42.461946 26553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:42.469302 26553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:42.469432 26553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:42.469601 26553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:43:42.479867 26553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:39983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39519
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:42.481204 26553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:42.482580 26553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:42.497272 26563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:42.497298 26559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:42.497510 26560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:42.501109 26553 server_base.cc:1034] running on GCE node
I20260808 02:43:42.501948 26553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:42.503367 26553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:42.504710 26553 hybrid_clock.cc:648] HybridClock initialized: now 1786157022504622 us; error 66 us; skew 500 ppm
I20260808 02:43:42.505180 26553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:42 dist-test-slave-k5rr krb5kdc[26167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157022, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:42.511514 26553 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:43:42.513278 26553 webserver.cc:460] Webserver started at http://127.25.137.194:36313/ using document root <none> and password file <none>
I20260808 02:43:42.514225 26553 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:42.514364 26553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:42.514780 26553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:42.518754 26553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "ab1fafe6fff04e17a5eb6941442b6c64"
format_stamp: "Formatted at 2026-08-08 02:43:42 on dist-test-slave-k5rr"
server_key: "698e46c72283017c371a23e8ce6039ff"
server_key_iv: "f99c2de28b99de8ff36813e368cca280"
server_key_version: "encryptionkey@0"
I20260808 02:43:42.519912 26553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "ab1fafe6fff04e17a5eb6941442b6c64"
format_stamp: "Formatted at 2026-08-08 02:43:42 on dist-test-slave-k5rr"
server_key: "698e46c72283017c371a23e8ce6039ff"
server_key_iv: "f99c2de28b99de8ff36813e368cca280"
server_key_version: "encryptionkey@0"
I20260808 02:43:42.527196 26553 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260808 02:43:42.531833 26576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:42.533594 26553 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:43:42.533785 26553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "ab1fafe6fff04e17a5eb6941442b6c64"
format_stamp: "Formatted at 2026-08-08 02:43:42 on dist-test-slave-k5rr"
server_key: "698e46c72283017c371a23e8ce6039ff"
server_key_iv: "f99c2de28b99de8ff36813e368cca280"
server_key_version: "encryptionkey@0"
I20260808 02:43:42.534011 26553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:42.549190 26553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:42.553364 26553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:42.553638 26553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:42.554668 26553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:43:42.555987 26553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:42.556067 26553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:42.556195 26553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:42.556249 26553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:42.590183 26553 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:39595
I20260808 02:43:42.590256 26759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:39595 every 8 connection(s)
I20260808 02:43:42.591439 26553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:43:42.599992 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 26553
I20260808 02:43:42.600200 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:43:42.600670 26151 external_mini_cluster.cc:1428] Setting key 43a46ced08a92b561d3009c2e44a13d5
Aug 08 02:43:42 dist-test-slave-k5rr krb5kdc[26167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157022, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:42.612886 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:42.594196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:45827 (local address 127.25.137.254:39519)
0808 02:43:42.594410 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:42.594417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:42.595698 (+ 1281us) server_negotiation.cc:407] Connection header received
0808 02:43:42.597664 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:42.597668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:42.597739 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:42.597835 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:42.599474 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:42.600224 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:42.602045 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:42.602600 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:42.602697 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:42.606461 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:42.606467 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:42.606473 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:42.606518 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:42.609641 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:42.610395 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:42.610400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:42.610403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:42.610546 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:42.611150 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:42.611155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:42.611158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:42.611546 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:42.611695 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:42.612450 (+ 755us) server_negotiation.cc:299] Negotiation successful
0808 02:43:42.612630 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":66}
I20260808 02:43:42.614240 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:42.594715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:39519 (local address 127.25.137.194:45827)
0808 02:43:42.595529 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:42.595576 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:42.597408 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:42.598077 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:42.598085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:42.598675 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:42.599306 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:42.599328 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:42.600401 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:42.600405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:42.601903 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:42.601919 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:42.602846 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:42.602854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:42.603075 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:42.603772 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:42.603805 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:42.606213 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:42.609825 (+ 3612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:42.609832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:42.609844 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:42.610261 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:42.610682 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:42.610685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:42.610688 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:42.610990 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:42.611678 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:42.611684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:42.612053 (+ 369us) client_negotiation.cc:769] Sending connection context
0808 02:43:42.612361 (+ 308us) client_negotiation.cc:240] Negotiation successful
0808 02:43:42.612909 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":320,"threads_started":1}
I20260808 02:43:42.615654 26760 heartbeater.cc:347] Connected to a master server at 127.25.137.254:39519
I20260808 02:43:42.615974 26760 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:42.616729 26760 heartbeater.cc:511] Master 127.25.137.254:39519 requested a full tablet report, sending...
I20260808 02:43:42.618316 26236 ts_manager.cc:194] Registered new tserver with Master: ab1fafe6fff04e17a5eb6941442b6c64 (127.25.137.194:39595)
I20260808 02:43:42.619187 26236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:45827
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:43:42.647595 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39519
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:39983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:42.985239 26770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:42.985558 26770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:42.985715 26770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:42.993394 26770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:42.993624 26770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:42.993830 26770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:43:43.004199 26770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:39983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39519
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:43.005610 26770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:43.006963 26770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:43.020258 26777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:43.020287 26780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:43.020331 26778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:43.021028 26770 server_base.cc:1034] running on GCE node
I20260808 02:43:43.021631 26770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:43.022619 26770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:43.023833 26770 hybrid_clock.cc:648] HybridClock initialized: now 1786157023023774 us; error 48 us; skew 500 ppm
I20260808 02:43:43.024139 26770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:43 dist-test-slave-k5rr krb5kdc[26167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157023, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:43.027456 26770 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:43:43.028995 26770 webserver.cc:460] Webserver started at http://127.25.137.195:38465/ using document root <none> and password file <none>
I20260808 02:43:43.029696 26770 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:43.029815 26770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:43.030110 26770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:43.033046 26770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "4113feebbbf443fd874dedc694b6440e"
format_stamp: "Formatted at 2026-08-08 02:43:43 on dist-test-slave-k5rr"
server_key: "0a4b4f56d9a8249b3eac046d19faf8f1"
server_key_iv: "456995a9845f55584d3b6a193ad28dba"
server_key_version: "encryptionkey@0"
I20260808 02:43:43.033818 26770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "4113feebbbf443fd874dedc694b6440e"
format_stamp: "Formatted at 2026-08-08 02:43:43 on dist-test-slave-k5rr"
server_key: "0a4b4f56d9a8249b3eac046d19faf8f1"
server_key_iv: "456995a9845f55584d3b6a193ad28dba"
server_key_version: "encryptionkey@0"
I20260808 02:43:43.039115 26770 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:43:43.042457 26791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:43.043787 26770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:43:43.043938 26770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "4113feebbbf443fd874dedc694b6440e"
format_stamp: "Formatted at 2026-08-08 02:43:43 on dist-test-slave-k5rr"
server_key: "0a4b4f56d9a8249b3eac046d19faf8f1"
server_key_iv: "456995a9845f55584d3b6a193ad28dba"
server_key_version: "encryptionkey@0"
I20260808 02:43:43.044122 26770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:43.065883 26770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:43.070107 26770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:43.070391 26770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:43.071360 26770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:43:43.072762 26770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:43.072849 26770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:43.072969 26770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:43.073042 26770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 02:43:43.108654 26770 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:34079
I20260808 02:43:43.108731 26960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:34079 every 8 connection(s)
I20260808 02:43:43.110061 26770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:43:43.112356 26541 heartbeater.cc:503] Master 127.25.137.254:39519 was elected leader, sending a full tablet report...
I20260808 02:43:43.116488 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 26770
I20260808 02:43:43.116698 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:43:43.117230 26151 external_mini_cluster.cc:1428] Setting key 2061657cf3820eb114862e4733d0d2db
Aug 08 02:43:43 dist-test-slave-k5rr krb5kdc[26167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157023, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:43.131217 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:43.113116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:51553 (local address 127.25.137.254:39519)
0808 02:43:43.113335 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:43.113341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:43.114643 (+ 1302us) server_negotiation.cc:407] Connection header received
0808 02:43:43.116605 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:43.116612 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:43.116704 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:43.116841 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:43.118492 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.119538 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.121302 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.121713 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.121826 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:43.125386 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:43.125390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:43.125394 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:43.125425 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:43.128127 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.128844 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.128850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.128854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.128998 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.129451 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.129455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.129457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.129801 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:43.129951 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:43.130647 (+ 696us) server_negotiation.cc:299] Negotiation successful
0808 02:43:43.130870 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:43:43.132215 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:43.113632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:39519 (local address 127.25.137.195:51553)
0808 02:43:43.114478 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:43.114529 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:43.116316 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:43.117139 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:43.117147 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:43.117714 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:43.118301 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:43.118319 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.119692 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.119695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:43.121148 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:43.121164 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.121935 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.121943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:43.122167 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:43.122813 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:43.122837 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:43.125162 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:43.128265 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:43.128274 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:43.128291 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:43.128712 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:43.129130 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:43.129136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:43.129140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:43.129331 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:43.129933 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:43.129941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:43.130271 (+ 330us) client_negotiation.cc:769] Sending connection context
0808 02:43:43.130512 (+ 241us) client_negotiation.cc:240] Negotiation successful
0808 02:43:43.131043 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":360,"threads_started":1}
I20260808 02:43:43.133728 26961 heartbeater.cc:347] Connected to a master server at 127.25.137.254:39519
I20260808 02:43:43.134145 26961 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:43.134882 26961 heartbeater.cc:511] Master 127.25.137.254:39519 requested a full tablet report, sending...
I20260808 02:43:43.136289 26236 ts_manager.cc:194] Registered new tserver with Master: 4113feebbbf443fd874dedc694b6440e (127.25.137.195:34079)
I20260808 02:43:43.137236 26236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:51553
I20260808 02:43:43.146476 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:43:43 dist-test-slave-k5rr krb5kdc[26167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157023, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:43:43 dist-test-slave-k5rr krb5kdc[26167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157023, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:43:43.198136 26970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:43.182478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.25.137.193:33873)
0808 02:43:43.183137 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:43.183147 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:43.183179 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:43:43.183349 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:43.183356 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:43.183571 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:43.183724 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:43.185294 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.186340 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.187621 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.188173 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.188289 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:43.192255 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:43.192268 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:43.192282 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:43.192317 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:43.195366 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.195996 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.196003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.196005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.196176 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.196561 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.196565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.196567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.196924 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:43.197234 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:43.197541 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 02:43:43.197806 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":274,"threads_started":1}
W20260808 02:43:43.199458 26407 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35710
Aug 08 02:43:43 dist-test-slave-k5rr krb5kdc[26167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:43:43 dist-test-slave-k5rr krb5kdc[26167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:43:43.245373 26970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:43.232336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35718 (local address 127.25.137.193:33873)
0808 02:43:43.232578 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:43.232585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:43.232825 (+ 240us) server_negotiation.cc:407] Connection header received
0808 02:43:43.232969 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:43.232973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:43.233072 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:43.233155 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:43.234489 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.235231 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.236442 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.236873 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.236967 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:43.240651 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:43.240656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:43.240659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:43.240692 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:43.243138 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.243712 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.243716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.243719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.243880 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.244148 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.244151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.244153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.244398 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:43.244575 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:43.244852 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 02:43:43.245058 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 02:43:43.247246 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26336
I20260808 02:43:43.259395 26970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:43.248474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.25.137.193:33873)
0808 02:43:43.248624 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:43.248627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:43.248971 (+ 344us) server_negotiation.cc:407] Connection header received
0808 02:43:43.249164 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:43.249168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:43.249223 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:43.249301 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:43.250404 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.251092 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.252332 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.252742 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.252839 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:43.254927 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:43.254932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:43.254935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:43.254962 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:43.257371 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.257954 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.257957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.257960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.258077 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.258347 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.258350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.258352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.258571 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:43.258732 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:43.258961 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 02:43:43.259107 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:43:43.260226 26535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:43:43.348691 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 26336
I20260808 02:43:43.370410 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26553
Aug 08 02:43:43 dist-test-slave-k5rr krb5kdc[26167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:43:43.386543 26980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:43.371303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.25.137.194:39595)
0808 02:43:43.371871 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:43.371878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:43.371903 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:43:43.371972 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:43.371978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:43.372189 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:43.372324 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:43.373552 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.374678 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.375897 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.376536 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.376642 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:43.380649 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:43.380662 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:43.380676 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:43.380712 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:43.383777 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.384389 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.384396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.384399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.384541 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.384893 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.384897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.384899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.385450 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:43.385697 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:43.385916 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 02:43:43.386123 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":314,"threads_started":1}
I20260808 02:43:43.387830 26752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:43:43.477362 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 26553
I20260808 02:43:43.498378 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26770
Aug 08 02:43:43 dist-test-slave-k5rr krb5kdc[26167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:43:43.514458 26984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:43.499470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49890 (local address 127.25.137.195:34079)
0808 02:43:43.500214 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:43.500221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:43.500252 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:43:43.500316 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:43.500323 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:43.500544 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:43.500670 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:43.502020 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.503136 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.504460 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:43.505092 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:43.505211 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:43.509252 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:43.509264 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:43.509279 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:43.509321 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:43.511895 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.512554 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.512562 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.512564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.512689 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:43.512974 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:43.512977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:43.512979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:43.513431 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:43.513669 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:43.513957 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 02:43:43.514160 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":417,"threads_started":1}
I20260808 02:43:43.515874 26954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:43:43.598611 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 26770
I20260808 02:43:43.618927 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26186
I20260808 02:43:43.620172 26284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:43:43.723867 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 26186
2026-08-08T02:43:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4390 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:43:43 dist-test-slave-k5rr krb5kdc[26990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:43:43 dist-test-slave-k5rr krb5kdc[26990](info): set up 2 sockets
Aug 08 02:43:43 dist-test-slave-k5rr krb5kdc[26990](info): commencing operation
krb5kdc: starting...
W20260808 02:43:44.933063 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:43:45 dist-test-slave-k5rr krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:43:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:43:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:43:45.126474 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:44071
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:36935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:44071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:45.443847 27018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:45.444159 27018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:45.444310 27018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:45.451718 27018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:43:45.451856 27018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:45.451921 27018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:45.452013 27018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:43:45.452090 27018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:43:45.462255 27018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:36935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:44071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:44071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:45.463567 27018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:45.464902 27018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:45.476542 27033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:45.476615 27028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:45.476593 27029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:45.481485 27018 server_base.cc:1034] running on GCE node
I20260808 02:43:45.482393 27018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:45.484016 27018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:45.485242 27018 hybrid_clock.cc:648] HybridClock initialized: now 1786157025485141 us; error 84 us; skew 500 ppm
I20260808 02:43:45.485591 27018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:45 dist-test-slave-k5rr krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157025, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:45.492650 27018 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:43:45.494232 27018 webserver.cc:460] Webserver started at http://127.25.137.254:39611/ using document root <none> and password file <none>
I20260808 02:43:45.494928 27018 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:45.495057 27018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:45.495386 27018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:45.498507 27018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "675b7c03c1144824a309b4f3a348d740"
format_stamp: "Formatted at 2026-08-08 02:43:45 on dist-test-slave-k5rr"
server_key: "4021c2d2e38bd9ebc9c40a1f85a76158"
server_key_iv: "4f2caf80fce037d5534130b0bc9d991a"
server_key_version: "encryptionkey@0"
I20260808 02:43:45.499323 27018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "675b7c03c1144824a309b4f3a348d740"
format_stamp: "Formatted at 2026-08-08 02:43:45 on dist-test-slave-k5rr"
server_key: "4021c2d2e38bd9ebc9c40a1f85a76158"
server_key_iv: "4f2caf80fce037d5534130b0bc9d991a"
server_key_version: "encryptionkey@0"
I20260808 02:43:45.504534 27018 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 02:43:45.507867 27044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:45.509164 27018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:43:45.509317 27018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "675b7c03c1144824a309b4f3a348d740"
format_stamp: "Formatted at 2026-08-08 02:43:45 on dist-test-slave-k5rr"
server_key: "4021c2d2e38bd9ebc9c40a1f85a76158"
server_key_iv: "4f2caf80fce037d5534130b0bc9d991a"
server_key_version: "encryptionkey@0"
I20260808 02:43:45.509502 27018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:45.532757 27018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:45.536764 27018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:45.537070 27018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:45.554760 27018 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:44071
I20260808 02:43:45.554780 27126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:44071 every 8 connection(s)
I20260808 02:43:45.556119 27018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:43:45.559860 27128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:45.564282 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 27018
I20260808 02:43:45.564538 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:43:45.565424 26151 external_mini_cluster.cc:1428] Setting key 6a0be8f8c9a1f3c1e3ee2035af8d4b72
I20260808 02:43:45.571592 27128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740: Bootstrap starting.
I20260808 02:43:45.575127 27128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:45.576282 27128 log.cc:824] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:45.578989 27128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740: No bootstrap required, opened a new log
Aug 08 02:43:45 dist-test-slave-k5rr krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:45.588871 27133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:45.567537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.25.137.254:44071)
0808 02:43:45.568352 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:45.568363 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:45.568410 (+ 47us) server_negotiation.cc:407] Connection header received
0808 02:43:45.570246 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:45.570269 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:45.570664 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:45.570982 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:45.572833 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:45.574246 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:45.575899 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:45.576604 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:45.576764 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:45.580534 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:45.580545 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:45.580567 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:45.580609 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:45.584493 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:45.585159 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:45.585171 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:45.585180 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:45.585389 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:45.585738 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:45.585743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:45.585746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:45.586418 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:45.586726 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:45.587123 (+ 397us) server_negotiation.cc:299] Negotiation successful
0808 02:43:45.587572 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":398,"threads_started":1}
I20260808 02:43:45.589148 27128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675b7c03c1144824a309b4f3a348d740" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44071 } }
I20260808 02:43:45.589598 27128 raft_consensus.cc:374] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:45.589712 27128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 675b7c03c1144824a309b4f3a348d740, State: Initialized, Role: FOLLOWER
I20260808 02:43:45.590319 27128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675b7c03c1144824a309b4f3a348d740" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44071 } }
I20260808 02:43:45.590562 27128 raft_consensus.cc:388] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:43:45.590698 27128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:43:45.590859 27128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:45.593273 27128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675b7c03c1144824a309b4f3a348d740" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44071 } }
I20260808 02:43:45.593714 27128 leader_election.cc:302] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 675b7c03c1144824a309b4f3a348d740; no voters:
I20260808 02:43:45.594283 27128 leader_election.cc:288] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:43:45.594426 27135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:43:45.595515 27135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [term 1 LEADER]: Becoming Leader. State: Replica: 675b7c03c1144824a309b4f3a348d740, State: Running, Role: LEADER
I20260808 02:43:45.596042 27135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675b7c03c1144824a309b4f3a348d740" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44071 } }
I20260808 02:43:45.596175 27128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:43:45.598819 27141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 675b7c03c1144824a309b4f3a348d740. Latest consensus state: current_term: 1 leader_uuid: "675b7c03c1144824a309b4f3a348d740" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675b7c03c1144824a309b4f3a348d740" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44071 } } }
I20260808 02:43:45.598917 27139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "675b7c03c1144824a309b4f3a348d740" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675b7c03c1144824a309b4f3a348d740" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44071 } } }
I20260808 02:43:45.599371 27141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [sys.catalog]: This master's current role is: LEADER
I20260808 02:43:45.599372 27139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740 [sys.catalog]: This master's current role is: LEADER
I20260808 02:43:45.600392 27152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:43:45.605918 27152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:43:45.616358 27152 catalog_manager.cc:1347] Generated new cluster ID: 8a9a21c9c37343669733b3e759676e91
I20260808 02:43:45.616461 27152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:43:45.638537 27152 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:43:45.639837 27152 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:43:45.654699 27152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 675b7c03c1144824a309b4f3a348d740: Generated new TSK 0
I20260808 02:43:45.655560 27152 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:43:45.714870 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:44071
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:36935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:46.045966 27169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:46.046228 27169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:46.046355 27169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:46.053503 27169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:46.053610 27169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:46.053756 27169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:43:46.063431 27169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:36935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:44071
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:46.064750 27169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:46.066174 27169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:46.078724 27178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:46.078837 27182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:46.078758 27185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:46.079885 27169 server_base.cc:1034] running on GCE node
I20260808 02:43:46.080447 27169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:46.081413 27169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:46.082619 27169 hybrid_clock.cc:648] HybridClock initialized: now 1786157026082574 us; error 40 us; skew 500 ppm
I20260808 02:43:46.082965 27169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:46 dist-test-slave-k5rr krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157026, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:46.086185 27169 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:43:46.087708 27169 webserver.cc:460] Webserver started at http://127.25.137.193:35805/ using document root <none> and password file <none>
I20260808 02:43:46.088377 27169 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:46.088502 27169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:46.088809 27169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:46.091744 27169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "eaac6ee36666451ab6160510eec62e8a"
format_stamp: "Formatted at 2026-08-08 02:43:46 on dist-test-slave-k5rr"
server_key: "514761c6edceff8fc5cdb3879a22126b"
server_key_iv: "ae7e5562632f2b3c3e6e80361483518a"
server_key_version: "encryptionkey@0"
I20260808 02:43:46.092545 27169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "eaac6ee36666451ab6160510eec62e8a"
format_stamp: "Formatted at 2026-08-08 02:43:46 on dist-test-slave-k5rr"
server_key: "514761c6edceff8fc5cdb3879a22126b"
server_key_iv: "ae7e5562632f2b3c3e6e80361483518a"
server_key_version: "encryptionkey@0"
I20260808 02:43:46.097600 27169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 02:43:46.101189 27194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:46.102550 27169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:43:46.102697 27169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "eaac6ee36666451ab6160510eec62e8a"
format_stamp: "Formatted at 2026-08-08 02:43:46 on dist-test-slave-k5rr"
server_key: "514761c6edceff8fc5cdb3879a22126b"
server_key_iv: "ae7e5562632f2b3c3e6e80361483518a"
server_key_version: "encryptionkey@0"
I20260808 02:43:46.102928 27169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:46.132548 27169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:46.136598 27169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:46.136881 27169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:46.137935 27169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:43:46.139261 27169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:46.139341 27169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:46.139465 27169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:46.139528 27169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:46.170878 27169 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:43003
I20260808 02:43:46.170964 27384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:43003 every 8 connection(s)
I20260808 02:43:46.172173 27169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:43:46.176682 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 27169
I20260808 02:43:46.176882 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:43:46.177532 26151 external_mini_cluster.cc:1428] Setting key 7b6d4becc7e4d5a5efe799adb0083841
Aug 08 02:43:46 dist-test-slave-k5rr krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157026, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:46.191962 27387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:46.174741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:39857 (local address 127.25.137.254:44071)
0808 02:43:46.175161 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:46.175166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:46.176120 (+ 954us) server_negotiation.cc:407] Connection header received
0808 02:43:46.178145 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:46.178151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:46.178238 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:46.178335 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:46.179796 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:46.180503 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:46.182041 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:46.182440 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:46.182551 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:46.185996 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:46.186002 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:46.186007 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:46.186044 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:46.189092 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:46.189791 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:46.189796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:46.189799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:46.189935 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:46.190323 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:46.190328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:46.190330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:46.190642 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:46.190803 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:46.191414 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 02:43:46.191630 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":210,"threads_started":1}
I20260808 02:43:46.192708 27389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:46.175289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:44071 (local address 127.25.137.193:39857)
0808 02:43:46.175966 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:46.176010 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:46.177883 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:46.178579 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:46.178587 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:46.179085 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:46.179623 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:46.179640 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:46.180655 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:46.180659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:46.181904 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:46.181917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:46.182654 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:46.182662 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:46.182880 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:46.183488 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:46.183510 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:46.185770 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:46.189223 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:46.189230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:46.189242 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:46.189644 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:46.190028 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:46.190031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:46.190034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:46.190212 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:46.190743 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:46.190748 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:46.191050 (+ 302us) client_negotiation.cc:769] Sending connection context
0808 02:43:46.191258 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 02:43:46.191630 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":275,"threads_started":1}
I20260808 02:43:46.194221 27385 heartbeater.cc:347] Connected to a master server at 127.25.137.254:44071
I20260808 02:43:46.194500 27385 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:46.195165 27385 heartbeater.cc:511] Master 127.25.137.254:44071 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
I20260808 02:43:46.197314 27073 ts_manager.cc:194] Registered new tserver with Master: eaac6ee36666451ab6160510eec62e8a (127.25.137.193:43003)
I20260808 02:43:46.198920 27073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:39857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:43:46.221662 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:44071
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:36935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:46.543944 27394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:46.544229 27394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:46.544353 27394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:46.551540 27394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:46.551671 27394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:46.551836 27394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:43:46.561683 27394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:36935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:44071
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:46.562965 27394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:46.564267 27394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:46.576694 27400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:46.576794 27401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:46.576762 27403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:46.577723 27394 server_base.cc:1034] running on GCE node
I20260808 02:43:46.578310 27394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:46.579200 27394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:46.580408 27394 hybrid_clock.cc:648] HybridClock initialized: now 1786157026580360 us; error 38 us; skew 500 ppm
I20260808 02:43:46.580710 27394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:46 dist-test-slave-k5rr krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157026, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:46.583971 27394 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:43:46.585489 27394 webserver.cc:460] Webserver started at http://127.25.137.194:43741/ using document root <none> and password file <none>
I20260808 02:43:46.586203 27394 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:46.586324 27394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:46.586692 27394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:46.589649 27394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "6748d3824c744a768ee42ad18a9809b9"
format_stamp: "Formatted at 2026-08-08 02:43:46 on dist-test-slave-k5rr"
server_key: "3ef69681a6d60100a6ffad87eb734cdc"
server_key_iv: "bbc484f5690e393fc2c43cc8ce0d8dc9"
server_key_version: "encryptionkey@0"
I20260808 02:43:46.590440 27394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "6748d3824c744a768ee42ad18a9809b9"
format_stamp: "Formatted at 2026-08-08 02:43:46 on dist-test-slave-k5rr"
server_key: "3ef69681a6d60100a6ffad87eb734cdc"
server_key_iv: "bbc484f5690e393fc2c43cc8ce0d8dc9"
server_key_version: "encryptionkey@0"
I20260808 02:43:46.595602 27394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:43:46.598965 27411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:46.600293 27394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:43:46.600435 27394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "6748d3824c744a768ee42ad18a9809b9"
format_stamp: "Formatted at 2026-08-08 02:43:46 on dist-test-slave-k5rr"
server_key: "3ef69681a6d60100a6ffad87eb734cdc"
server_key_iv: "bbc484f5690e393fc2c43cc8ce0d8dc9"
server_key_version: "encryptionkey@0"
I20260808 02:43:46.600610 27394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:46.619788 27394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:46.623950 27394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:46.624221 27394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:46.625222 27394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:43:46.626534 27394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:46.626611 27394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:46.626732 27394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:46.626785 27394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:46.657169 27394 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:33651
I20260808 02:43:46.657203 27589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:33651 every 8 connection(s)
I20260808 02:43:46.658411 27394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:43:46.661039 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 27394
I20260808 02:43:46.661269 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:43:46.661944 26151 external_mini_cluster.cc:1428] Setting key 14dcbcab8cfc2b2a8cd587adc15966f6
Aug 08 02:43:46 dist-test-slave-k5rr krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157026, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:46.679402 27387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:46.661166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:57839 (local address 127.25.137.254:44071)
0808 02:43:46.661374 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:46.661380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:46.662890 (+ 1510us) server_negotiation.cc:407] Connection header received
0808 02:43:46.664811 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:46.664825 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:46.664911 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:46.664993 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:46.666512 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:46.667214 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:46.668731 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:46.669222 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:46.669331 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:46.673174 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:46.673180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:46.673183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:46.673216 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:46.675971 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:46.676823 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:46.676830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:46.676833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:46.676995 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:46.677514 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:46.677519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:46.677523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:46.677890 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:46.678061 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:46.678791 (+ 730us) server_negotiation.cc:299] Negotiation successful
0808 02:43:46.679034 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:43:46.680367 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:46.661855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:44071 (local address 127.25.137.194:57839)
0808 02:43:46.662705 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:46.662754 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:46.664557 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:46.665258 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:46.665266 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:46.665782 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:46.666346 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:46.666365 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:46.667370 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:46.667374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:46.668591 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:46.668605 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:46.669465 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:46.669472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:46.669673 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:46.670327 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:46.670365 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:46.672897 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:46.676130 (+ 3233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:46.676140 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:46.676158 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:46.676657 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:46.677153 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:46.677156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:46.677159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:46.677377 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:46.678034 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:46.678047 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:46.678397 (+ 350us) client_negotiation.cc:769] Sending connection context
0808 02:43:46.678645 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 02:43:46.679123 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":352,"threads_started":1}
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
I20260808 02:43:46.681936 27592 heartbeater.cc:347] Connected to a master server at 127.25.137.254:44071
I20260808 02:43:46.682221 27592 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:46.682942 27592 heartbeater.cc:511] Master 127.25.137.254:44071 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
I20260808 02:43:46.684597 27073 ts_manager.cc:194] Registered new tserver with Master: 6748d3824c744a768ee42ad18a9809b9 (127.25.137.194:33651)
I20260808 02:43:46.685724 27073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:57839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:43:46.707762 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:44071
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:36935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:47.039199 27602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:47.039485 27602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:47.039582 27602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:47.046851 27602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:47.046963 27602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:47.047080 27602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:43:47.056799 27602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:36935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:44071
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:47.058063 27602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:47.059367 27602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:47.072228 27608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:47.072180 27609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:47.072258 27612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:47.072906 27602 server_base.cc:1034] running on GCE node
I20260808 02:43:47.073553 27602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:47.074532 27602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:47.075745 27602 hybrid_clock.cc:648] HybridClock initialized: now 1786157027075682 us; error 55 us; skew 500 ppm
I20260808 02:43:47.076056 27602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:47 dist-test-slave-k5rr krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157027, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:47.079131 27602 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:43:47.080611 27602 webserver.cc:460] Webserver started at http://127.25.137.195:41881/ using document root <none> and password file <none>
I20260808 02:43:47.081326 27602 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:47.081449 27602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:47.081745 27602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:47.084661 27602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "27a252fb3fb943fb8b93f4650cb57d60"
format_stamp: "Formatted at 2026-08-08 02:43:47 on dist-test-slave-k5rr"
server_key: "6de97fb3d0b95775a44a08582a868112"
server_key_iv: "4b74936e0fb9fbd3a1c5353993b84122"
server_key_version: "encryptionkey@0"
I20260808 02:43:47.085492 27602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "27a252fb3fb943fb8b93f4650cb57d60"
format_stamp: "Formatted at 2026-08-08 02:43:47 on dist-test-slave-k5rr"
server_key: "6de97fb3d0b95775a44a08582a868112"
server_key_iv: "4b74936e0fb9fbd3a1c5353993b84122"
server_key_version: "encryptionkey@0"
I20260808 02:43:47.090566 27602 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:43:47.093986 27620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:47.095227 27602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:43:47.095381 27602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "27a252fb3fb943fb8b93f4650cb57d60"
format_stamp: "Formatted at 2026-08-08 02:43:47 on dist-test-slave-k5rr"
server_key: "6de97fb3d0b95775a44a08582a868112"
server_key_iv: "4b74936e0fb9fbd3a1c5353993b84122"
server_key_version: "encryptionkey@0"
I20260808 02:43:47.095561 27602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:47.114988 27602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:47.119367 27602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:47.119649 27602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:47.120641 27602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:43:47.122067 27602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:47.122170 27602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:47.122309 27602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:47.122364 27602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:47.154069 27602 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:41169
I20260808 02:43:47.154126 27793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:41169 every 8 connection(s)
I20260808 02:43:47.155335 27602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:43:47.157588 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 27602
I20260808 02:43:47.157769 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:43:47.158272 26151 external_mini_cluster.cc:1428] Setting key 47c35599fa937d5f8e60227200acab38
Aug 08 02:43:47 dist-test-slave-k5rr krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157027, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:47.174984 27387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.157812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:34607 (local address 127.25.137.254:44071)
0808 02:43:47.158028 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.158034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.159261 (+ 1227us) server_negotiation.cc:407] Connection header received
0808 02:43:47.161106 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.161111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.161179 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.161255 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:47.162717 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.163472 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.165048 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.165462 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.165562 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.169071 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:47.169076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:47.169079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:47.169119 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:47.171795 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.172630 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.172635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.172639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.172792 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.173314 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.173320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.173323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.173656 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:47.173802 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.174564 (+ 762us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.174735 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 02:43:47.176290 27797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.158288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:44071 (local address 127.25.137.195:34607)
0808 02:43:47.159109 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:47.159153 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:47.160855 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:47.161451 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:47.161459 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:47.162013 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:47.162563 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.162581 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.163617 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.163621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.164876 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.164890 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.165677 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.165683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.165964 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.166559 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:47.166585 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:47.168831 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:47.171975 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:47.171984 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:47.171998 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:47.172504 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:47.172917 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:47.172922 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:47.172925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:47.173203 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:47.173779 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:47.173786 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:47.174186 (+ 400us) client_negotiation.cc:769] Sending connection context
0808 02:43:47.174469 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 02:43:47.174973 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":320,"threads_started":1}
I20260808 02:43:47.177589 27794 heartbeater.cc:347] Connected to a master server at 127.25.137.254:44071
I20260808 02:43:47.177877 27794 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:47.178597 27794 heartbeater.cc:511] Master 127.25.137.254:44071 requested a full tablet report, sending...
I20260808 02:43:47.179961 27073 ts_manager.cc:194] Registered new tserver with Master: 27a252fb3fb943fb8b93f4650cb57d60 (127.25.137.195:41169)
I20260808 02:43:47.180712 27073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:34607
I20260808 02:43:47.186268 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:43:47.202046 27385 heartbeater.cc:503] Master 127.25.137.254:44071 was elected leader, sending a full tablet report...
I20260808 02:43:47.206511 27387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.193541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.25.137.254:44071)
0808 02:43:47.193704 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.193709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.194207 (+ 498us) server_negotiation.cc:407] Connection header received
0808 02:43:47.194370 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.194374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.194437 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.194522 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:47.195991 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.196732 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.198163 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.198624 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.198755 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.201041 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:47.201046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:47.201049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:47.201080 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:47.204032 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.204727 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.204731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.204733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.204904 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.205333 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.205336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.205338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.205624 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:47.205898 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.206078 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.206255 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:43:47.215106 27073 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:43:47.223310 27073 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:43:47.271819 27811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.240343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38654 (local address 127.25.137.194:33651)
0808 02:43:47.263969 (+ 23626us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.263977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.263999 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:43:47.264077 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.264083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.264253 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.264431 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:47.265880 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.267296 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.269239 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.270456 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.270612 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.270774 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.271244 (+ 470us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.271430 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23408,"thread_start_us":23308,"threads_started":1}
I20260808 02:43:47.271994 27812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.240435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:41169 (local address 127.0.0.1:52268)
0808 02:43:47.261421 (+ 20986us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:47.261443 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:47.261580 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:47.262423 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:47.262434 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:47.262475 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:47.262933 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.262948 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.264966 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.264972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.267601 (+ 2629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.267618 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.270942 (+ 3324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.270949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.271167 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.271203 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:43:47.271307 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 02:43:47.271455 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20855,"mutex_wait_us":33,"thread_start_us":20696,"threads_started":1}
I20260808 02:43:47.272374 27810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.240090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:33651 (local address 127.0.0.1:38654)
0808 02:43:47.256381 (+ 16291us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:47.256447 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:47.256774 (+ 327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:47.264674 (+ 7900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:47.264678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:47.264704 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:47.265667 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.265683 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.267429 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.267433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.269091 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.269104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.270679 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.270686 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.270906 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.271011 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 02:43:47.271170 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 02:43:47.271398 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15910,"mutex_wait_us":375,"thread_start_us":15711,"threads_started":1}
I20260808 02:43:47.272848 27814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.240881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:43003 (local address 127.0.0.1:47788)
0808 02:43:47.263568 (+ 22687us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:47.263587 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:47.263703 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:47.264440 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:47.264446 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:47.264475 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:47.264994 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.265037 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.267066 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.267073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.269586 (+ 2513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.269602 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.271735 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.271741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.272057 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.272088 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:43:47.272166 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 02:43:47.272292 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22561,"mutex_wait_us":5,"thread_start_us":22418,"threads_started":1}
I20260808 02:43:47.272967 27818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.241570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47788 (local address 127.25.137.193:43003)
0808 02:43:47.263125 (+ 21555us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.263134 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.263835 (+ 701us) server_negotiation.cc:407] Connection header received
0808 02:43:47.263980 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.263986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.264225 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.264422 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:47.265262 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.266924 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.269750 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.271234 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.271424 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.271590 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.272489 (+ 899us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.272639 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21382,"thread_start_us":21205,"threads_started":1}
I20260808 02:43:47.272979 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.241126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52268 (local address 127.25.137.195:41169)
0808 02:43:47.260739 (+ 19613us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.260750 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.261719 (+ 969us) server_negotiation.cc:407] Connection header received
0808 02:43:47.261839 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.261848 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.262156 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.262361 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:47.263215 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.264781 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.267826 (+ 3045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.269587 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.269808 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.269987 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.272457 (+ 2470us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.272631 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19396,"thread_start_us":19621,"threads_started":1}
I20260808 02:43:47.277165 27277 tablet_service.cc:1467] Processing CreateTablet for tablet f64e64efeeb2442a836242693d621bbd (DEFAULT_TABLE table=test-table [id=a2d3593190924749a600515ac5ef87ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:47.278602 27277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64e64efeeb2442a836242693d621bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:47.279057 27491 tablet_service.cc:1467] Processing CreateTablet for tablet f64e64efeeb2442a836242693d621bbd (DEFAULT_TABLE table=test-table [id=a2d3593190924749a600515ac5ef87ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:47.279116 27694 tablet_service.cc:1467] Processing CreateTablet for tablet f64e64efeeb2442a836242693d621bbd (DEFAULT_TABLE table=test-table [id=a2d3593190924749a600515ac5ef87ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:47.280251 27491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64e64efeeb2442a836242693d621bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:47.280484 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64e64efeeb2442a836242693d621bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:47.292793 27820 tablet_bootstrap.cc:492] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9: Bootstrap starting.
I20260808 02:43:47.297056 27820 tablet_bootstrap.cc:654] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:47.298138 27820 log.cc:824] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:47.299645 27821 tablet_bootstrap.cc:492] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a: Bootstrap starting.
I20260808 02:43:47.301285 27820 tablet_bootstrap.cc:492] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9: No bootstrap required, opened a new log
I20260808 02:43:47.301568 27820 ts_tablet_manager.cc:1397] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 02:43:47.303445 27821 tablet_bootstrap.cc:654] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:47.304432 27821 log.cc:824] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:47.305292 27822 tablet_bootstrap.cc:492] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60: Bootstrap starting.
I20260808 02:43:47.307453 27821 tablet_bootstrap.cc:492] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a: No bootstrap required, opened a new log
I20260808 02:43:47.307704 27821 ts_tablet_manager.cc:1397] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260808 02:43:47.308228 27822 tablet_bootstrap.cc:654] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:47.309185 27822 log.cc:824] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:47.311559 27822 tablet_bootstrap.cc:492] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60: No bootstrap required, opened a new log
I20260808 02:43:47.311779 27822 ts_tablet_manager.cc:1397] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.004s
I20260808 02:43:47.311581 27820 raft_consensus.cc:348] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6748d3824c744a768ee42ad18a9809b9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33651 } } peers { permanent_uuid: "27a252fb3fb943fb8b93f4650cb57d60" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41169 } } peers { permanent_uuid: "eaac6ee36666451ab6160510eec62e8a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43003 } }
I20260808 02:43:47.312033 27820 raft_consensus.cc:374] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:47.312131 27820 raft_consensus.cc:729] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6748d3824c744a768ee42ad18a9809b9, State: Initialized, Role: FOLLOWER
I20260808 02:43:47.312798 27820 consensus_queue.cc:260] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6748d3824c744a768ee42ad18a9809b9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33651 } } peers { permanent_uuid: "27a252fb3fb943fb8b93f4650cb57d60" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41169 } } peers { permanent_uuid: "eaac6ee36666451ab6160510eec62e8a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43003 } }
I20260808 02:43:47.314198 27592 heartbeater.cc:503] Master 127.25.137.254:44071 was elected leader, sending a full tablet report...
I20260808 02:43:47.314585 27820 ts_tablet_manager.cc:1428] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260808 02:43:47.316910 27821 raft_consensus.cc:348] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6748d3824c744a768ee42ad18a9809b9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33651 } } peers { permanent_uuid: "27a252fb3fb943fb8b93f4650cb57d60" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41169 } } peers { permanent_uuid: "eaac6ee36666451ab6160510eec62e8a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43003 } }
I20260808 02:43:47.317287 27821 raft_consensus.cc:374] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:47.317380 27821 raft_consensus.cc:729] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaac6ee36666451ab6160510eec62e8a, State: Initialized, Role: FOLLOWER
I20260808 02:43:47.317930 27821 consensus_queue.cc:260] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6748d3824c744a768ee42ad18a9809b9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33651 } } peers { permanent_uuid: "27a252fb3fb943fb8b93f4650cb57d60" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41169 } } peers { permanent_uuid: "eaac6ee36666451ab6160510eec62e8a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43003 } }
I20260808 02:43:47.319535 27821 ts_tablet_manager.cc:1428] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260808 02:43:47.321384 27822 raft_consensus.cc:348] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6748d3824c744a768ee42ad18a9809b9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33651 } } peers { permanent_uuid: "27a252fb3fb943fb8b93f4650cb57d60" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41169 } } peers { permanent_uuid: "eaac6ee36666451ab6160510eec62e8a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43003 } }
I20260808 02:43:47.321832 27822 raft_consensus.cc:374] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:47.321950 27822 raft_consensus.cc:729] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a252fb3fb943fb8b93f4650cb57d60, State: Initialized, Role: FOLLOWER
I20260808 02:43:47.322556 27822 consensus_queue.cc:260] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6748d3824c744a768ee42ad18a9809b9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33651 } } peers { permanent_uuid: "27a252fb3fb943fb8b93f4650cb57d60" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41169 } } peers { permanent_uuid: "eaac6ee36666451ab6160510eec62e8a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43003 } }
I20260808 02:43:47.323563 27794 heartbeater.cc:503] Master 127.25.137.254:44071 was elected leader, sending a full tablet report...
I20260808 02:43:47.324047 27822 ts_tablet_manager.cc:1428] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 02:43:47.339146 27828 raft_consensus.cc:482] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:43:47.339370 27828 raft_consensus.cc:504] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6748d3824c744a768ee42ad18a9809b9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33651 } } peers { permanent_uuid: "27a252fb3fb943fb8b93f4650cb57d60" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41169 } } peers { permanent_uuid: "eaac6ee36666451ab6160510eec62e8a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43003 } }
I20260808 02:43:47.341414 27828 leader_election.cc:288] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6748d3824c744a768ee42ad18a9809b9 (127.25.137.194:33651), 27a252fb3fb943fb8b93f4650cb57d60 (127.25.137.195:41169)
I20260808 02:43:47.347808 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.341921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:43907 (local address 127.25.137.195:41169)
0808 02:43:47.342114 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.342120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.342651 (+ 531us) server_negotiation.cc:407] Connection header received
0808 02:43:47.342823 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.342827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.342882 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.342954 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:47.343583 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.344315 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.346192 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.346942 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.347084 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.347154 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.347449 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.347567 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:43:47.347867 27811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.341717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:50725 (local address 127.25.137.194:33651)
0808 02:43:47.342008 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.342015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.342762 (+ 747us) server_negotiation.cc:407] Connection header received
0808 02:43:47.342921 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.342925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.342980 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.343051 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:47.343706 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.344418 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.346224 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.346972 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.347113 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.347183 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.347505 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.347625 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260808 02:43:47.348080 27832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.341717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:41169 (local address 127.25.137.193:43907)
0808 02:43:47.342553 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:47.342568 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:47.342701 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:47.342986 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:47.342991 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:47.343013 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:47.343434 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.343443 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.344446 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.344451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.346048 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.346061 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.347175 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.347182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.347302 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.347325 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:43:47.347402 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 02:43:47.347518 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":60,"spinlock_wait_cycles":25472,"thread_start_us":501,"threads_started":1}
I20260808 02:43:47.348104 27834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.341797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:33651 (local address 127.25.137.193:50725)
0808 02:43:47.342654 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:47.342668 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:47.342787 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:47.343082 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:47.343086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:47.343187 (+ 101us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:47.343546 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.343554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.344554 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.344559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.346077 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:47.346090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.347243 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.347250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:47.347363 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.347385 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:43:47.347462 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 02:43:47.347576 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":39,"spinlock_wait_cycles":19840,"thread_start_us":515,"threads_started":1}
I20260808 02:43:47.349148 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64e64efeeb2442a836242693d621bbd" candidate_uuid: "eaac6ee36666451ab6160510eec62e8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a252fb3fb943fb8b93f4650cb57d60" is_pre_election: true
I20260808 02:43:47.349156 27520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64e64efeeb2442a836242693d621bbd" candidate_uuid: "eaac6ee36666451ab6160510eec62e8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6748d3824c744a768ee42ad18a9809b9" is_pre_election: true
I20260808 02:43:47.349779 27719 raft_consensus.cc:2454] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaac6ee36666451ab6160510eec62e8a in term 0.
I20260808 02:43:47.350629 27200 leader_election.cc:302] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a252fb3fb943fb8b93f4650cb57d60, eaac6ee36666451ab6160510eec62e8a; no voters:
I20260808 02:43:47.351025 27828 raft_consensus.cc:2781] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:43:47.351284 27828 raft_consensus.cc:482] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:43:47.351395 27828 raft_consensus.cc:3037] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:47.351997 27520 raft_consensus.cc:2454] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaac6ee36666451ab6160510eec62e8a in term 0.
I20260808 02:43:47.354005 27828 raft_consensus.cc:504] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6748d3824c744a768ee42ad18a9809b9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33651 } } peers { permanent_uuid: "27a252fb3fb943fb8b93f4650cb57d60" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41169 } } peers { permanent_uuid: "eaac6ee36666451ab6160510eec62e8a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43003 } }
I20260808 02:43:47.354593 27828 leader_election.cc:288] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [CANDIDATE]: Term 1 election: Requested vote from peers 6748d3824c744a768ee42ad18a9809b9 (127.25.137.194:33651), 27a252fb3fb943fb8b93f4650cb57d60 (127.25.137.195:41169)
I20260808 02:43:47.355032 27520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64e64efeeb2442a836242693d621bbd" candidate_uuid: "eaac6ee36666451ab6160510eec62e8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6748d3824c744a768ee42ad18a9809b9"
I20260808 02:43:47.355077 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64e64efeeb2442a836242693d621bbd" candidate_uuid: "eaac6ee36666451ab6160510eec62e8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a252fb3fb943fb8b93f4650cb57d60"
I20260808 02:43:47.355242 27719 raft_consensus.cc:3037] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:47.355242 27520 raft_consensus.cc:3037] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:47.358407 27520 raft_consensus.cc:2454] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaac6ee36666451ab6160510eec62e8a in term 1.
I20260808 02:43:47.358399 27719 raft_consensus.cc:2454] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaac6ee36666451ab6160510eec62e8a in term 1.
I20260808 02:43:47.358804 27200 leader_election.cc:302] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a252fb3fb943fb8b93f4650cb57d60, eaac6ee36666451ab6160510eec62e8a; no voters:
I20260808 02:43:47.359102 27828 raft_consensus.cc:2781] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:43:47.359596 27828 raft_consensus.cc:686] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [term 1 LEADER]: Becoming Leader. State: Replica: eaac6ee36666451ab6160510eec62e8a, State: Running, Role: LEADER
I20260808 02:43:47.360143 27828 consensus_queue.cc:237] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6748d3824c744a768ee42ad18a9809b9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33651 } } peers { permanent_uuid: "27a252fb3fb943fb8b93f4650cb57d60" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41169 } } peers { permanent_uuid: "eaac6ee36666451ab6160510eec62e8a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43003 } }
I20260808 02:43:47.365320 27070 catalog_manager.cc:5495] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a reported cstate change: term changed from 0 to 1, leader changed from <none> to eaac6ee36666451ab6160510eec62e8a (127.25.137.193). New cstate: current_term: 1 leader_uuid: "eaac6ee36666451ab6160510eec62e8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6748d3824c744a768ee42ad18a9809b9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27a252fb3fb943fb8b93f4650cb57d60" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaac6ee36666451ab6160510eec62e8a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43003 } health_report { overall_health: HEALTHY } } }
Aug 08 02:43:47 dist-test-slave-k5rr krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:43:47.396014 27818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.381954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.25.137.193:43003)
0808 02:43:47.382120 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.382125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.382646 (+ 521us) server_negotiation.cc:407] Connection header received
0808 02:43:47.382786 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.382790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.382859 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.382950 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:47.384474 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.385211 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.386464 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.386852 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.386950 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.390930 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:47.390945 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:47.390960 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:47.390993 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:47.393664 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.394276 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.394283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.394286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.394431 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.394697 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.394700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.394702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.395042 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:47.395252 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.395480 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.395673 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 02:43:47.406852 27795 tablet.cc:2369] T f64e64efeeb2442a836242693d621bbd P 27a252fb3fb943fb8b93f4650cb57d60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:43:47.410409 27594 tablet.cc:2369] T f64e64efeeb2442a836242693d621bbd P 6748d3824c744a768ee42ad18a9809b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:43:47.424624 27386 tablet.cc:2369] T f64e64efeeb2442a836242693d621bbd P eaac6ee36666451ab6160510eec62e8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 02:43:47 dist-test-slave-k5rr krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157027, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:43:47 dist-test-slave-k5rr krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157027, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:43:47.447068 27818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.433655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47802 (local address 127.25.137.193:43003)
0808 02:43:47.433905 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.433912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.434181 (+ 269us) server_negotiation.cc:407] Connection header received
0808 02:43:47.434363 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.434367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.434437 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.434522 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:47.435972 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.436696 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.437969 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.438410 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.438517 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.442232 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:47.442237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:47.442241 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:47.442276 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:47.444647 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.445268 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.445272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.445274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.445397 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.445727 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.445730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.445732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.446028 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:47.446246 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.446528 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.446723 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 02:43:47 dist-test-slave-k5rr krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:43:47 dist-test-slave-k5rr krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:43:47.496382 27818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.481648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47816 (local address 127.25.137.193:43003)
0808 02:43:47.481917 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.481923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.482226 (+ 303us) server_negotiation.cc:407] Connection header received
0808 02:43:47.482430 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.482435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.482534 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.482647 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:47.484393 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.485367 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.486630 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.487056 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.487150 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.491116 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:47.491121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:47.491124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:47.491160 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:47.493755 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.494664 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.494668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.494671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.494820 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.495160 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.495163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.495165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.495454 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:47.495647 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.495922 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.496108 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 02:43:47.498829 27246 tablet_service.cc:2908] Rejecting scan request for tablet f64e64efeeb2442a836242693d621bbd: Uninitialized: clean time has not yet been initialized
I20260808 02:43:47.503185 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27169
I20260808 02:43:47.515873 27818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.504022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47824 (local address 127.25.137.193:43003)
0808 02:43:47.504210 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.504217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.504567 (+ 350us) server_negotiation.cc:407] Connection header received
0808 02:43:47.504711 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.504717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.504842 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.504930 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:47.506119 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.506882 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.508099 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.508506 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.508602 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.510773 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:47.510778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:47.510781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:47.510860 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:47.513409 (+ 2549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.514152 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.514155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.514157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.514297 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.514607 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.514609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.514611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.514852 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:47.515030 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.515335 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.515493 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:43:47.516571 27373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:43:47.608198 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 27169
I20260808 02:43:47.632601 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27394
Aug 08 02:43:47 dist-test-slave-k5rr krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:43:47.646677 27811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.633525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38668 (local address 127.25.137.194:33651)
0808 02:43:47.633685 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.633690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.633706 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:43:47.633764 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.633768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.633841 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.633920 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:47.635065 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.635897 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.637190 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.637606 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.637699 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.641480 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:47.641494 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:47.641510 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:47.641542 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:47.644158 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.644733 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.644739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.644741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.644907 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.645366 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.645370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.645372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.645751 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:47.646045 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.646270 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.646426 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:43:47.647338 27583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:43:47.734315 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 27394
I20260808 02:43:47.751081 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27602
Aug 08 02:43:47 dist-test-slave-k5rr krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:43:47.764993 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:47.751835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52284 (local address 127.25.137.195:41169)
0808 02:43:47.752002 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:47.752007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:47.752024 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:43:47.752170 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:47.752174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:47.752249 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:47.752335 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:47.753623 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.754369 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.755634 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:47.756034 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:47.756158 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:47.760014 (+ 3856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:47.760027 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:47.760043 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:47.760082 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:47.762829 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.763392 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.763398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.763400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.763527 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:47.763821 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:47.763824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:47.763826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:47.764157 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:47.764377 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:47.764567 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 02:43:47.764726 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:43:47.765630 27787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:43:47.852994 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 27602
I20260808 02:43:47.877096 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27018
I20260808 02:43:47.878237 27120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:43:47.986622 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 27018
2026-08-08T02:43:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4266 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:43:48 dist-test-slave-k5rr krb5kdc[27867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:43:48 dist-test-slave-k5rr krb5kdc[27867](info): set up 2 sockets
Aug 08 02:43:48 dist-test-slave-k5rr krb5kdc[27867](info): commencing operation
krb5kdc: starting...
W20260808 02:43:49.191248 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:43:49 dist-test-slave-k5rr krb5kdc[27867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:43:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:43:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:43:49.385605 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:33315
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:33315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:49.710345 27884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:49.710700 27884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:49.710867 27884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:49.718470 27884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:43:49.718614 27884 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:43:49.718696 27884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:49.718811 27884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:49.718890 27884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:43:49.718987 27884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:43:49.729409 27884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:33315
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:33315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:49.730748 27884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:49.732126 27884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:49.743884 27891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:49.743844 27899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:49.743937 27894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:49.745131 27884 server_base.cc:1034] running on GCE node
I20260808 02:43:49.745867 27884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:49.747225 27884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:49.748449 27884 hybrid_clock.cc:648] HybridClock initialized: now 1786157029748385 us; error 37 us; skew 500 ppm
I20260808 02:43:49.748756 27884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:49 dist-test-slave-k5rr krb5kdc[27867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157029, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:49.751772 27884 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:43:49.753185 27884 webserver.cc:460] Webserver started at http://127.25.137.254:46377/ using document root <none> and password file <none>
I20260808 02:43:49.753824 27884 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:49.753919 27884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:49.754186 27884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:49.757026 27884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "dd9696fe4a7a480f855fa25f8dc34ddb"
format_stamp: "Formatted at 2026-08-08 02:43:49 on dist-test-slave-k5rr"
server_key: "6677b5f90222988585dcdc160c668ee1"
server_key_iv: "3c72a15bbc323d3ee8437d80ae85990c"
server_key_version: "encryptionkey@0"
I20260808 02:43:49.757781 27884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "dd9696fe4a7a480f855fa25f8dc34ddb"
format_stamp: "Formatted at 2026-08-08 02:43:49 on dist-test-slave-k5rr"
server_key: "6677b5f90222988585dcdc160c668ee1"
server_key_iv: "3c72a15bbc323d3ee8437d80ae85990c"
server_key_version: "encryptionkey@0"
I20260808 02:43:49.762713 27884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:43:49.765949 27908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:49.767235 27884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:43:49.767398 27884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "dd9696fe4a7a480f855fa25f8dc34ddb"
format_stamp: "Formatted at 2026-08-08 02:43:49 on dist-test-slave-k5rr"
server_key: "6677b5f90222988585dcdc160c668ee1"
server_key_iv: "3c72a15bbc323d3ee8437d80ae85990c"
server_key_version: "encryptionkey@0"
I20260808 02:43:49.767601 27884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:49.780351 27884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:49.784245 27884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:49.784490 27884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:49.804723 27884 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:33315
I20260808 02:43:49.804762 27990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:33315 every 8 connection(s)
I20260808 02:43:49.806069 27884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:43:49.809661 27991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:49.813916 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 27884
I20260808 02:43:49.814148 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:43:49.814956 26151 external_mini_cluster.cc:1428] Setting key 4c5d9fd32808b2afaff6f63c264ca4cb
I20260808 02:43:49.821039 27991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb: Bootstrap starting.
I20260808 02:43:49.824515 27991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:49.825657 27991 log.cc:824] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:49.828227 27991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb: No bootstrap required, opened a new log
Aug 08 02:43:49 dist-test-slave-k5rr krb5kdc[27867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:49.835800 27991 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9696fe4a7a480f855fa25f8dc34ddb" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33315 } }
I20260808 02:43:49.836138 27991 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:49.836237 27991 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9696fe4a7a480f855fa25f8dc34ddb, State: Initialized, Role: FOLLOWER
I20260808 02:43:49.836880 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:49.816919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34434 (local address 127.25.137.254:33315)
0808 02:43:49.817668 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:49.817678 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:49.817721 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:43:49.819622 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:49.819642 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:49.820006 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:49.820305 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:49.822041 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:49.823378 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:49.825080 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:49.825745 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:49.825936 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:49.829769 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:49.829777 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:49.829790 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:49.829836 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:49.832814 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:49.833485 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:49.833491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:49.833497 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:49.833672 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:49.834031 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:49.834034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:49.834036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:49.834571 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:49.834874 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:49.835291 (+ 417us) server_negotiation.cc:299] Negotiation successful
0808 02:43:49.835676 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":321,"threads_started":1}
I20260808 02:43:49.836772 27991 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9696fe4a7a480f855fa25f8dc34ddb" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33315 } }
I20260808 02:43:49.837148 27991 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:43:49.837275 27991 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:43:49.837453 27991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:49.839761 27991 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9696fe4a7a480f855fa25f8dc34ddb" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33315 } }
I20260808 02:43:49.840234 27991 leader_election.cc:302] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd9696fe4a7a480f855fa25f8dc34ddb; no voters:
I20260808 02:43:49.840729 27991 leader_election.cc:288] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:43:49.840842 28003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:43:49.841509 28003 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [term 1 LEADER]: Becoming Leader. State: Replica: dd9696fe4a7a480f855fa25f8dc34ddb, State: Running, Role: LEADER
I20260808 02:43:49.841956 28003 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9696fe4a7a480f855fa25f8dc34ddb" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33315 } }
I20260808 02:43:49.842621 27991 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:43:49.844480 28005 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd9696fe4a7a480f855fa25f8dc34ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9696fe4a7a480f855fa25f8dc34ddb" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33315 } } }
I20260808 02:43:49.844928 28005 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [sys.catalog]: This master's current role is: LEADER
I20260808 02:43:49.845175 28007 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd9696fe4a7a480f855fa25f8dc34ddb. Latest consensus state: current_term: 1 leader_uuid: "dd9696fe4a7a480f855fa25f8dc34ddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9696fe4a7a480f855fa25f8dc34ddb" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33315 } } }
I20260808 02:43:49.845487 28007 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb [sys.catalog]: This master's current role is: LEADER
I20260808 02:43:49.845824 28018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:43:49.851792 28018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:43:49.859802 28018 catalog_manager.cc:1347] Generated new cluster ID: f0cd720ecff44c2694a2801c63548565
I20260808 02:43:49.859927 28018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:43:49.884842 28018 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:43:49.886206 28018 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:43:49.893656 28018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb: Generated new TSK 0
I20260808 02:43:49.894686 28018 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:43:49.947021 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:33315
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:50.264796 28035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:50.265094 28035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:50.265236 28035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:50.272374 28035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:50.272490 28035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:50.272616 28035 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:43:50.272728 28035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:43:50.282351 28035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.137.254:33315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:50.283649 28035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:50.285077 28035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:50.297690 28046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:50.297694 28047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:50.297720 28049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:50.298839 28035 server_base.cc:1034] running on GCE node
I20260808 02:43:50.299383 28035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:50.300226 28035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:50.301422 28035 hybrid_clock.cc:648] HybridClock initialized: now 1786157030301354 us; error 45 us; skew 500 ppm
I20260808 02:43:50.301692 28035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:50 dist-test-slave-k5rr krb5kdc[27867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:50.304692 28035 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:43:50.306114 28035 webserver.cc:460] Webserver started at http://127.25.137.193:37349/ using document root <none> and password file <none>
I20260808 02:43:50.306758 28035 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:50.306859 28035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:50.307161 28035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:50.309996 28035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "d7d74c26a4af454189d5f9a2039a52a5"
format_stamp: "Formatted at 2026-08-08 02:43:50 on dist-test-slave-k5rr"
server_key: "9493b2431bde4e7f7edc5c143331f5a5"
server_key_iv: "a054faa2cfda0a8335a6183c9d14faa1"
server_key_version: "encryptionkey@0"
I20260808 02:43:50.310767 28035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "d7d74c26a4af454189d5f9a2039a52a5"
format_stamp: "Formatted at 2026-08-08 02:43:50 on dist-test-slave-k5rr"
server_key: "9493b2431bde4e7f7edc5c143331f5a5"
server_key_iv: "a054faa2cfda0a8335a6183c9d14faa1"
server_key_version: "encryptionkey@0"
I20260808 02:43:50.315740 28035 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:43:50.319015 28062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:50.320302 28035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:43:50.320477 28035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "d7d74c26a4af454189d5f9a2039a52a5"
format_stamp: "Formatted at 2026-08-08 02:43:50 on dist-test-slave-k5rr"
server_key: "9493b2431bde4e7f7edc5c143331f5a5"
server_key_iv: "a054faa2cfda0a8335a6183c9d14faa1"
server_key_version: "encryptionkey@0"
I20260808 02:43:50.320696 28035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:50.338210 28035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:50.342662 28035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:50.342939 28035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:50.345839 28035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:50.345942 28035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:50.346079 28035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:50.346158 28035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:43:50 dist-test-slave-k5rr krb5kdc[27867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:50.370114 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:50.346398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:44193 (local address 127.25.137.254:33315)
0808 02:43:50.348595 (+ 2197us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:50.348601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:50.348623 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:43:50.350565 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:50.350572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:50.350662 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:50.350787 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:50.353460 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.354466 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.357398 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.357968 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.358100 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:50.363082 (+ 4982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:50.363088 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:50.363093 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:50.363139 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:50.366193 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:50.367203 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:50.367209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:50.367212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:50.367367 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:50.367941 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:50.367946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:50.367949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:50.368281 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:50.368449 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:50.369522 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0808 02:43:50.369747 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2074,"thread_start_us":1968,"threads_started":1}
I20260808 02:43:50.371585 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:50.346969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:33315 (local address 127.25.137.193:44193)
0808 02:43:50.347879 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:50.347924 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:50.350239 (+ 2315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:50.351285 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:50.351309 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:50.352205 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:50.353228 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:50.353263 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.354663 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.354669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:50.357221 (+ 2552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:50.357237 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.358452 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.358461 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:50.358789 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:50.359616 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:50.359655 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:50.362779 (+ 3124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:50.366373 (+ 3594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:50.366380 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:50.366392 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:50.367035 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:50.367476 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:50.367481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:50.367485 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:50.367770 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:50.368528 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:50.368544 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:50.369115 (+ 571us) client_negotiation.cc:769] Sending connection context
0808 02:43:50.369406 (+ 291us) client_negotiation.cc:240] Negotiation successful
0808 02:43:50.369981 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":348,"threads_started":1}
I20260808 02:43:50.385039 28035 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:33139
I20260808 02:43:50.385113 28253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:33139 every 8 connection(s)
I20260808 02:43:50.386312 28035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:43:50.386933 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 28035
I20260808 02:43:50.387163 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:43:50.388031 26151 external_mini_cluster.cc:1428] Setting key beb9986931f4645554f6763e191bdf8f
I20260808 02:43:50.388422 28255 heartbeater.cc:347] Connected to a master server at 127.25.137.254:33315
I20260808 02:43:50.388638 28255 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:50.389503 28255 heartbeater.cc:511] Master 127.25.137.254:33315 requested a full tablet report, sending...
I20260808 02:43:50.391882 27929 ts_manager.cc:194] Registered new tserver with Master: d7d74c26a4af454189d5f9a2039a52a5 (127.25.137.193:33139)
I20260808 02:43:50.392693 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:50.379175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:33315 (local address 127.25.137.193:53191)
0808 02:43:50.379735 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:50.379750 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:50.379900 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:50.380415 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:50.380420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:50.380883 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:50.381270 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:50.381282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.382556 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.382562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:50.383809 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:50.383830 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.384667 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.384674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:50.384851 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:50.385427 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:50.385444 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:50.386162 (+ 718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:50.389705 (+ 3543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:50.389709 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:50.389713 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:50.390226 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:50.390737 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:50.390742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:50.390744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:50.390925 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:50.391616 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:50.391622 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:50.391814 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 02:43:50.391949 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 02:43:50.392259 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":305,"threads_started":1}
I20260808 02:43:50.393980 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:50.378955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:53191 (local address 127.25.137.254:33315)
0808 02:43:50.379132 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:50.379136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:50.379827 (+ 691us) server_negotiation.cc:407] Connection header received
0808 02:43:50.380022 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:50.380027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:50.380108 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:50.380196 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:50.381413 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.382346 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.383948 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.384472 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.384578 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:50.386342 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:50.386347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:50.386350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:50.386379 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:50.389556 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:50.390396 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:50.390401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:50.390405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:50.390567 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:50.391075 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:50.391079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:50.391082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:50.391467 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:50.392948 (+ 1481us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:50.393103 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 02:43:50.393442 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:43:50.394726 27929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:44193
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:43:50.433604 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:33315
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:50.756270 28263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:50.756574 28263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:50.756731 28263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:50.764189 28263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:50.764339 28263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:50.764484 28263 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:43:50.764612 28263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:43:50.774906 28263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.137.254:33315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:50.776214 28263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:50.777696 28263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:50.790707 28274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:50.790642 28273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:50.790747 28277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:50.791512 28263 server_base.cc:1034] running on GCE node
I20260808 02:43:50.792140 28263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:50.793035 28263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:50.794261 28263 hybrid_clock.cc:648] HybridClock initialized: now 1786157030794197 us; error 37 us; skew 500 ppm
I20260808 02:43:50.794556 28263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:50 dist-test-slave-k5rr krb5kdc[27867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:50.799993 28263 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:43:50.801407 28263 webserver.cc:460] Webserver started at http://127.25.137.194:45287/ using document root <none> and password file <none>
I20260808 02:43:50.802055 28263 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:50.802146 28263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:50.802405 28263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:50.805318 28263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "a48476a3e6a4470d9f0c49c3c2948a13"
format_stamp: "Formatted at 2026-08-08 02:43:50 on dist-test-slave-k5rr"
server_key: "ca9b9cb3d23289ff28362706697cf485"
server_key_iv: "dabf6605013e95ea8c4c697405884aab"
server_key_version: "encryptionkey@0"
I20260808 02:43:50.806094 28263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "a48476a3e6a4470d9f0c49c3c2948a13"
format_stamp: "Formatted at 2026-08-08 02:43:50 on dist-test-slave-k5rr"
server_key: "ca9b9cb3d23289ff28362706697cf485"
server_key_iv: "dabf6605013e95ea8c4c697405884aab"
server_key_version: "encryptionkey@0"
I20260808 02:43:50.811254 28263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:43:50.814829 28284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:50.816205 28263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:43:50.816351 28263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "a48476a3e6a4470d9f0c49c3c2948a13"
format_stamp: "Formatted at 2026-08-08 02:43:50 on dist-test-slave-k5rr"
server_key: "ca9b9cb3d23289ff28362706697cf485"
server_key_iv: "dabf6605013e95ea8c4c697405884aab"
server_key_version: "encryptionkey@0"
I20260808 02:43:50.816546 28263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:50.836215 28263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:50.840327 28263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:50.840576 28263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:50.843276 28263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:50.843364 28263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:50.843533 28263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:50.843616 28263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:43:50 dist-test-slave-k5rr krb5kdc[27867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:50.865222 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:50.843620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:33207 (local address 127.25.137.254:33315)
0808 02:43:50.843782 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:50.843794 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:50.845211 (+ 1417us) server_negotiation.cc:407] Connection header received
0808 02:43:50.847346 (+ 2135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:50.847352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:50.847435 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:50.847522 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:50.849540 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.850504 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.852881 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.853477 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.853595 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:50.858536 (+ 4941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:50.858542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:50.858545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:50.858579 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:50.861700 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:50.862690 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:50.862696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:50.862700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:50.862842 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:50.863400 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:50.863405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:50.863409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:50.863690 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:50.863842 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:50.864615 (+ 773us) server_negotiation.cc:299] Negotiation successful
0808 02:43:50.864845 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 02:43:50.866271 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:50.844164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:33315 (local address 127.25.137.194:33207)
0808 02:43:50.845032 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:50.845087 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:50.847031 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:50.847763 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:50.847773 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:50.848479 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:50.849362 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:50.849385 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.850646 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.850650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:50.852745 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:50.852759 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.853790 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.853801 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:50.854135 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:50.854877 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:50.854912 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:50.858297 (+ 3385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:50.861893 (+ 3596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:50.861905 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:50.861922 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:50.862560 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:50.862991 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:50.863000 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:50.863004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:50.863277 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:50.863830 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:50.863839 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:50.864218 (+ 379us) client_negotiation.cc:769] Sending connection context
0808 02:43:50.864486 (+ 268us) client_negotiation.cc:240] Negotiation successful
0808 02:43:50.865050 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":335,"threads_started":1}
I20260808 02:43:50.891784 28263 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:39823
I20260808 02:43:50.891799 28482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:39823 every 8 connection(s)
I20260808 02:43:50.893525 28263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:43:50.896189 28483 heartbeater.cc:347] Connected to a master server at 127.25.137.254:33315
I20260808 02:43:50.896420 28483 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:50.897143 28483 heartbeater.cc:511] Master 127.25.137.254:33315 requested a full tablet report, sending...
I20260808 02:43:50.897375 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:50.883513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:39257 (local address 127.25.137.254:33315)
0808 02:43:50.883678 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:50.883682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:50.884314 (+ 632us) server_negotiation.cc:407] Connection header received
0808 02:43:50.884479 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:50.884485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:50.884586 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:50.884701 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:50.885797 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.886750 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.888695 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.889404 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.889550 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:50.891624 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:50.891629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:50.891633 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:50.891666 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:50.894349 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:50.895194 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:50.895199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:50.895203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:50.895362 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:50.895740 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:50.895745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:50.895749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:50.896215 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:50.896364 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:50.896776 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 02:43:50.896991 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:43:50.897513 28450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:50.883600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:33315 (local address 127.25.137.194:39257)
0808 02:43:50.884200 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:50.884219 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:50.884367 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:50.884712 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:50.884715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:50.885234 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:50.885634 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:50.885648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.886957 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.886964 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:50.888501 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:50.888523 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:50.889623 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:50.889630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:50.889809 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:50.890476 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:50.890493 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:50.891433 (+ 940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:50.894473 (+ 3040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:50.894479 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:50.894483 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:50.895042 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:50.895459 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:50.895464 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:50.895468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:50.895618 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:50.896341 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:50.896346 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:50.896532 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 02:43:50.896658 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 02:43:50.897043 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":337,"threads_started":1}
I20260808 02:43:50.898725 27928 ts_manager.cc:194] Registered new tserver with Master: a48476a3e6a4470d9f0c49c3c2948a13 (127.25.137.194:39823)
I20260808 02:43:50.899549 27928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:33207
I20260808 02:43:50.902521 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 28263
I20260808 02:43:50.902697 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:43:50.903115 26151 external_mini_cluster.cc:1428] Setting key e0b1b699f818a3d5021c0d2c4356deaf
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:43:50.944773 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:33315
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:51.264266 28487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:51.264535 28487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:51.264667 28487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:51.272028 28487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:51.272138 28487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:51.272246 28487 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:43:51.272351 28487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:43:51.282164 28487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.137.254:33315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:51.283463 28487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:51.284787 28487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:51.297756 28495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:51.297726 28497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:51.297794 28492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:51.298861 28487 server_base.cc:1034] running on GCE node
I20260808 02:43:51.299445 28487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:51.300340 28487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:51.301553 28487 hybrid_clock.cc:648] HybridClock initialized: now 1786157031301477 us; error 40 us; skew 500 ppm
I20260808 02:43:51.301846 28487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:51 dist-test-slave-k5rr krb5kdc[27867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157031, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:51.304879 28487 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:43:51.306255 28487 webserver.cc:460] Webserver started at http://127.25.137.195:39767/ using document root <none> and password file <none>
I20260808 02:43:51.306907 28487 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:51.306996 28487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:51.307257 28487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:51.310173 28487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "29026c9259604618be10e58225dae770"
format_stamp: "Formatted at 2026-08-08 02:43:51 on dist-test-slave-k5rr"
server_key: "5e31fca39e0fe8411588b325fdfc153d"
server_key_iv: "589ca8503d0c3c4f089ce59ee6870afe"
server_key_version: "encryptionkey@0"
I20260808 02:43:51.310925 28487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "29026c9259604618be10e58225dae770"
format_stamp: "Formatted at 2026-08-08 02:43:51 on dist-test-slave-k5rr"
server_key: "5e31fca39e0fe8411588b325fdfc153d"
server_key_iv: "589ca8503d0c3c4f089ce59ee6870afe"
server_key_version: "encryptionkey@0"
I20260808 02:43:51.316088 28487 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260808 02:43:51.319376 28509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:51.320665 28487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:43:51.320822 28487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "29026c9259604618be10e58225dae770"
format_stamp: "Formatted at 2026-08-08 02:43:51 on dist-test-slave-k5rr"
server_key: "5e31fca39e0fe8411588b325fdfc153d"
server_key_iv: "589ca8503d0c3c4f089ce59ee6870afe"
server_key_version: "encryptionkey@0"
I20260808 02:43:51.321050 28487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:51.333534 28487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:51.337523 28487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:51.337777 28487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:51.340307 28487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:51.340394 28487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:51.340519 28487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:51.340637 28487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:43:51 dist-test-slave-k5rr krb5kdc[27867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157031, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:51.362524 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.340782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:42107 (local address 127.25.137.254:33315)
0808 02:43:51.340996 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.341034 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.342247 (+ 1213us) server_negotiation.cc:407] Connection header received
0808 02:43:51.344365 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.344371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.344457 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.344541 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:51.346575 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.347589 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.349866 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.350447 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.350569 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.355499 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:51.355506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:51.355511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:51.355556 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:51.358748 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:51.359657 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:51.359663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:51.359667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:51.359843 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:51.360409 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:51.360412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:51.360414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:51.360678 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:51.360856 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.361879 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.362136 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:43:51.364913 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.341279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:33315 (local address 127.25.137.195:42107)
0808 02:43:51.342083 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:51.342128 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:51.344077 (+ 1949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:51.344775 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:51.344786 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:51.345495 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:51.346370 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.346401 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.347746 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.347750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.349697 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.349717 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.350705 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.350713 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.350991 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.351753 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:51.351806 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:51.355213 (+ 3407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:51.358898 (+ 3685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:51.358906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:51.358926 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:51.359534 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:51.359977 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:51.359988 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:51.359992 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:51.360288 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:51.360808 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:51.360818 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:51.361379 (+ 561us) client_negotiation.cc:769] Sending connection context
0808 02:43:51.361741 (+ 362us) client_negotiation.cc:240] Negotiation successful
0808 02:43:51.362360 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":334,"threads_started":1}
I20260808 02:43:51.383716 28487 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:44121
I20260808 02:43:51.383728 28697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:44121 every 8 connection(s)
I20260808 02:43:51.385218 28487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:43:51.387313 28700 heartbeater.cc:347] Connected to a master server at 127.25.137.254:33315
I20260808 02:43:51.387626 28700 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:51.387807 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.372026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:33315 (local address 127.25.137.195:47787)
0808 02:43:51.373477 (+ 1451us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:51.373495 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:51.373608 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:51.374030 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:51.374034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:51.374450 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:51.374764 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.374793 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.376097 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.376103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.377750 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.377764 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.378760 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.378768 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.378899 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.379546 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:51.379574 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:51.380563 (+ 989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:51.384334 (+ 3771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:51.384339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:51.384343 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:51.384853 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:51.385348 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:51.385353 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:51.385356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:51.385517 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:51.386563 (+ 1046us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:51.386570 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:51.386782 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 02:43:51.386932 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 02:43:51.387289 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1305,"thread_start_us":1163,"threads_started":1}
I20260808 02:43:51.388767 28700 heartbeater.cc:511] Master 127.25.137.254:33315 requested a full tablet report, sending...
I20260808 02:43:51.388880 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.372526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:47787 (local address 127.25.137.254:33315)
0808 02:43:51.372728 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.372735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.373564 (+ 829us) server_negotiation.cc:407] Connection header received
0808 02:43:51.373769 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.373775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.373863 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.373960 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:51.374928 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.375958 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.377900 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.378397 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.378518 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.380766 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:51.380774 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:51.380779 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:51.380830 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:51.384155 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:51.384979 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:51.384984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:51.384987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:51.385204 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:51.385642 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:51.385647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:51.385650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:51.386406 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:51.388102 (+ 1696us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.388249 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.388486 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:43:51.390854 27928 ts_manager.cc:194] Registered new tserver with Master: 29026c9259604618be10e58225dae770 (127.25.137.195:44121)
I20260808 02:43:51.391924 27928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:42107
I20260808 02:43:51.394610 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 28487
I20260808 02:43:51.394776 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:43:51.395236 26151 external_mini_cluster.cc:1428] Setting key 741bd689b425c26b3fa2990fd7d63f17
I20260808 02:43:51.398285 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:43:51.398326 28255 heartbeater.cc:503] Master 127.25.137.254:33315 was elected leader, sending a full tablet report...
Aug 08 02:43:51 dist-test-slave-k5rr krb5kdc[27867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157031, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:43:51 dist-test-slave-k5rr krb5kdc[27867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157031, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:51.448161 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.434111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34448 (local address 127.25.137.254:33315)
0808 02:43:51.434343 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.434350 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.434615 (+ 265us) server_negotiation.cc:407] Connection header received
0808 02:43:51.434801 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.434805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.434873 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.434956 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:51.436384 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.437173 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.438426 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.438833 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.438942 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.442978 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:51.442988 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:51.442992 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:51.443034 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:51.445591 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:51.446198 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:51.446204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:51.446206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:51.446341 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:51.446717 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:51.446721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:51.446723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:51.447024 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:51.447271 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.447572 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.447810 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:43:51.452455 27928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:43:51.459403 27928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:43:51.490950 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.471626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.25.137.194:39823)
0808 02:43:51.483245 (+ 11619us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.483254 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.483283 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:43:51.483392 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.483401 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.483631 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.483820 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:51.484659 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.486169 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.488824 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.489850 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.490021 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.490203 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.490481 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.490651 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9514,"thread_start_us":9368,"threads_started":1}
I20260808 02:43:51.491192 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.472126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:44121 (local address 127.0.0.1:38480)
0808 02:43:51.481649 (+ 9523us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:51.481667 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:51.481798 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:51.482605 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:51.482614 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:51.482649 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:51.483057 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.483070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.485609 (+ 2539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.485615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.488273 (+ 2658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.488301 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.490335 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.490342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.490531 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.490569 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 02:43:51.490634 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:43:51.490774 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9401,"mutex_wait_us":46,"thread_start_us":933,"threads_started":1}
I20260808 02:43:51.492074 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.471404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:39823 (local address 127.0.0.1:43800)
0808 02:43:51.472864 (+ 1460us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:51.472913 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:51.473117 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:51.484056 (+ 10939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:51.484061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:51.484088 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:51.484460 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.484475 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.486351 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.486356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.488618 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.488639 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.490046 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.490052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.490253 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.490310 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 02:43:51.490420 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 02:43:51.490618 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1179,"mutex_wait_us":5,"thread_start_us":9963,"threads_started":1}
I20260808 02:43:51.492249 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.472801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38480 (local address 127.25.137.195:44121)
0808 02:43:51.481078 (+ 8277us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.481088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.481933 (+ 845us) server_negotiation.cc:407] Connection header received
0808 02:43:51.482049 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.482057 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.482339 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.483302 (+ 963us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:51.483612 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.485438 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.488513 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.490068 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.491536 (+ 1468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.491714 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.491784 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.491945 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8067,"thread_start_us":7854,"threads_started":1}
I20260808 02:43:51.495838 28586 tablet_service.cc:1467] Processing CreateTablet for tablet ddee17b7b9234f0ca957e775e7e1b631 (DEFAULT_TABLE table=test-table [id=6bf3488c842540378efafcbd916ec301]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:51.495777 28371 tablet_service.cc:1467] Processing CreateTablet for tablet ddee17b7b9234f0ca957e775e7e1b631 (DEFAULT_TABLE table=test-table [id=6bf3488c842540378efafcbd916ec301]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:51.497181 28586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddee17b7b9234f0ca957e775e7e1b631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:51.497371 28371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddee17b7b9234f0ca957e775e7e1b631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:51.500627 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.487975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:33139 (local address 127.0.0.1:41728)
0808 02:43:51.488910 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:51.488929 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:51.489057 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:51.493503 (+ 4446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:51.493508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:51.493534 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:51.493858 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.493871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.495821 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.495827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.497774 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.497790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.499847 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.499853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.500033 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.500066 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:43:51.500152 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:43:51.500308 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":593,"threads_started":1}
I20260808 02:43:51.501500 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.482088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41728 (local address 127.25.137.193:33139)
0808 02:43:51.492933 (+ 10845us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.492941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.492967 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:43:51.493087 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.493095 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.493284 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.493512 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:51.494056 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.495671 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.497962 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.499390 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.499560 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.499711 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.501051 (+ 1340us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.501212 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10705,"thread_start_us":10578,"threads_started":1}
I20260808 02:43:51.505421 28137 tablet_service.cc:1467] Processing CreateTablet for tablet ddee17b7b9234f0ca957e775e7e1b631 (DEFAULT_TABLE table=test-table [id=6bf3488c842540378efafcbd916ec301]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:51.506829 28137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddee17b7b9234f0ca957e775e7e1b631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:51.508721 28732 tablet_bootstrap.cc:492] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: Bootstrap starting.
I20260808 02:43:51.510807 28734 tablet_bootstrap.cc:492] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: Bootstrap starting.
I20260808 02:43:51.512418 28732 tablet_bootstrap.cc:654] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:51.513787 28732 log.cc:824] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:51.514346 28734 tablet_bootstrap.cc:654] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:51.515472 28734 log.cc:824] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:51.516813 28732 tablet_bootstrap.cc:492] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: No bootstrap required, opened a new log
I20260808 02:43:51.517114 28732 ts_tablet_manager.cc:1397] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260808 02:43:51.518383 28734 tablet_bootstrap.cc:492] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: No bootstrap required, opened a new log
I20260808 02:43:51.518651 28734 ts_tablet_manager.cc:1397] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 02:43:51.525188 28738 tablet_bootstrap.cc:492] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: Bootstrap starting.
I20260808 02:43:51.528223 28738 tablet_bootstrap.cc:654] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:51.528012 28732 raft_consensus.cc:348] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.528474 28732 raft_consensus.cc:374] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:51.528587 28732 raft_consensus.cc:729] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29026c9259604618be10e58225dae770, State: Initialized, Role: FOLLOWER
I20260808 02:43:51.528899 28734 raft_consensus.cc:348] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.529338 28734 raft_consensus.cc:374] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:51.529238 28732 consensus_queue.cc:260] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.529435 28734 raft_consensus.cc:729] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48476a3e6a4470d9f0c49c3c2948a13, State: Initialized, Role: FOLLOWER
I20260808 02:43:51.529448 28738 log.cc:824] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:51.530058 28734 consensus_queue.cc:260] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.530871 28700 heartbeater.cc:503] Master 127.25.137.254:33315 was elected leader, sending a full tablet report...
I20260808 02:43:51.531383 28483 heartbeater.cc:503] Master 127.25.137.254:33315 was elected leader, sending a full tablet report...
I20260808 02:43:51.531524 28732 ts_tablet_manager.cc:1428] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260808 02:43:51.532601 28738 tablet_bootstrap.cc:492] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: No bootstrap required, opened a new log
I20260808 02:43:51.532979 28738 ts_tablet_manager.cc:1397] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 02:43:51.533078 28734 ts_tablet_manager.cc:1428] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260808 02:43:51.540884 28738 raft_consensus.cc:348] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.541252 28738 raft_consensus.cc:374] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:51.541350 28738 raft_consensus.cc:729] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d74c26a4af454189d5f9a2039a52a5, State: Initialized, Role: FOLLOWER
I20260808 02:43:51.541790 28738 consensus_queue.cc:260] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.543363 28738 ts_tablet_manager.cc:1428] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: Time spent starting tablet: real 0.010s user 0.006s sys 0.003s
I20260808 02:43:51.570139 28742 raft_consensus.cc:482] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:43:51.570415 28742 raft_consensus.cc:504] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.572198 28742 leader_election.cc:288] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a48476a3e6a4470d9f0c49c3c2948a13 (127.25.137.194:39823), 29026c9259604618be10e58225dae770 (127.25.137.195:44121)
I20260808 02:43:51.579365 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.573264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:55433 (local address 127.25.137.195:44121)
0808 02:43:51.573479 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.573486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.573730 (+ 244us) server_negotiation.cc:407] Connection header received
0808 02:43:51.573944 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.573949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.574026 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.574110 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:51.574789 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.575606 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.577685 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.578452 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.578599 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.578672 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.578964 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.579098 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 02:43:51.579397 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.572840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:56167 (local address 127.25.137.194:39823)
0808 02:43:51.573094 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.573101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.573954 (+ 853us) server_negotiation.cc:407] Connection header received
0808 02:43:51.574127 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.574132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.574205 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.574314 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:51.574949 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.575674 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.577683 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.578453 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.578600 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.578672 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.579024 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.579151 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:43:51.579509 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.572565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:44121 (local address 127.25.137.193:55433)
0808 02:43:51.573611 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:51.573631 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:51.573794 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:51.574133 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:51.574137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:51.574164 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:51.574630 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.574643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.575735 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.575740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.577527 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.577541 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.578670 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.578676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.578810 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.578835 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:43:51.578910 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:43:51.579027 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":887,"mutex_wait_us":61,"spinlock_wait_cycles":4224,"thread_start_us":650,"threads_started":1}
I20260808 02:43:51.579555 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.572623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:39823 (local address 127.25.137.193:56167)
0808 02:43:51.573850 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:51.573866 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:51.573993 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:51.574341 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:51.574345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:51.574386 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:51.574775 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.574798 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.575826 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.575830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.577525 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.577538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.578725 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.578733 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.578860 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.578883 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:43:51.578975 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 02:43:51.579097 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"mutex_wait_us":17,"spinlock_wait_cycles":3840,"thread_start_us":911,"threads_started":1}
I20260808 02:43:51.580597 28404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddee17b7b9234f0ca957e775e7e1b631" candidate_uuid: "d7d74c26a4af454189d5f9a2039a52a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" is_pre_election: true
I20260808 02:43:51.580673 28618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddee17b7b9234f0ca957e775e7e1b631" candidate_uuid: "d7d74c26a4af454189d5f9a2039a52a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29026c9259604618be10e58225dae770" is_pre_election: true
I20260808 02:43:51.581132 28404 raft_consensus.cc:2454] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d74c26a4af454189d5f9a2039a52a5 in term 0.
I20260808 02:43:51.581173 28618 raft_consensus.cc:2454] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d74c26a4af454189d5f9a2039a52a5 in term 0.
I20260808 02:43:51.581792 28064 leader_election.cc:302] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a48476a3e6a4470d9f0c49c3c2948a13, d7d74c26a4af454189d5f9a2039a52a5; no voters:
I20260808 02:43:51.582178 28742 raft_consensus.cc:2781] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:43:51.582310 28742 raft_consensus.cc:482] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:43:51.582386 28742 raft_consensus.cc:3037] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:51.584524 28742 raft_consensus.cc:504] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.585161 28742 leader_election.cc:288] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [CANDIDATE]: Term 1 election: Requested vote from peers a48476a3e6a4470d9f0c49c3c2948a13 (127.25.137.194:39823), 29026c9259604618be10e58225dae770 (127.25.137.195:44121)
I20260808 02:43:51.585508 28404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddee17b7b9234f0ca957e775e7e1b631" candidate_uuid: "d7d74c26a4af454189d5f9a2039a52a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48476a3e6a4470d9f0c49c3c2948a13"
I20260808 02:43:51.585641 28618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddee17b7b9234f0ca957e775e7e1b631" candidate_uuid: "d7d74c26a4af454189d5f9a2039a52a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29026c9259604618be10e58225dae770"
I20260808 02:43:51.585726 28404 raft_consensus.cc:3037] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:51.585852 28618 raft_consensus.cc:3037] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:51.588963 28404 raft_consensus.cc:2454] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d74c26a4af454189d5f9a2039a52a5 in term 1.
I20260808 02:43:51.588963 28618 raft_consensus.cc:2454] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d74c26a4af454189d5f9a2039a52a5 in term 1.
I20260808 02:43:51.589392 28064 leader_election.cc:302] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 29026c9259604618be10e58225dae770, a48476a3e6a4470d9f0c49c3c2948a13, d7d74c26a4af454189d5f9a2039a52a5; no voters:
I20260808 02:43:51.589681 28742 raft_consensus.cc:2781] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:43:51.590122 28742 raft_consensus.cc:686] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 1 LEADER]: Becoming Leader. State: Replica: d7d74c26a4af454189d5f9a2039a52a5, State: Running, Role: LEADER
I20260808 02:43:51.590572 28742 consensus_queue.cc:237] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.594921 27928 catalog_manager.cc:5495] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7d74c26a4af454189d5f9a2039a52a5 (127.25.137.193). New cstate: current_term: 1 leader_uuid: "d7d74c26a4af454189d5f9a2039a52a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } health_report { overall_health: UNKNOWN } } }
W20260808 02:43:51.641964 28701 tablet.cc:2369] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:43:51.646028 28484 tablet.cc:2369] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:43:51.650954 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.645320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41740 (local address 127.25.137.193:33139)
0808 02:43:51.645543 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.645550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.645670 (+ 120us) server_negotiation.cc:407] Connection header received
0808 02:43:51.646062 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.646067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.646167 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.646304 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:43:51.646676 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.647382 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.648674 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.649127 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.649274 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.649779 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:43:51.650347 (+ 568us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:43:51.650479 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.650612 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.650697 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260808 02:43:51.651886 28258 tablet.cc:2369] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:43:51.664070 28404 raft_consensus.cc:1261] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13 [term 1 FOLLOWER]: Refusing update from remote peer d7d74c26a4af454189d5f9a2039a52a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:43:51.665308 28618 raft_consensus.cc:1261] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770 [term 1 FOLLOWER]: Refusing update from remote peer d7d74c26a4af454189d5f9a2039a52a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:43:51.665306 28747 consensus_queue.cc:1035] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:43:51.666162 28742 consensus_queue.cc:1035] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:43:51.677503 28753 mvcc.cc:205] Tried to move back new op lower bound from 7316099201668243456 to 7316099201398517760. Current Snapshot: MvccSnapshot[applied={T|T < 7316099201668243456}]
I20260808 02:43:51.682927 28755 mvcc.cc:205] Tried to move back new op lower bound from 7316099201668243456 to 7316099201398517760. Current Snapshot: MvccSnapshot[applied={T|T < 7316099201668243456}]
I20260808 02:43:51.702733 27928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:43:51.703061 27928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:43:51.707347 27928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb: Sending DeleteTablet for 3 replicas of tablet ddee17b7b9234f0ca957e775e7e1b631
I20260808 02:43:51.708937 28371 tablet_service.cc:1514] Processing DeleteTablet for tablet ddee17b7b9234f0ca957e775e7e1b631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:43:51 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:43800
I20260808 02:43:51.708920 28137 tablet_service.cc:1514] Processing DeleteTablet for tablet ddee17b7b9234f0ca957e775e7e1b631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:43:51 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:41728
I20260808 02:43:51.709132 28586 tablet_service.cc:1514] Processing DeleteTablet for tablet ddee17b7b9234f0ca957e775e7e1b631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:43:51 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:38480
I20260808 02:43:51.712096 27928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:43:51.713022 27928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:43:51.723201 28585 tablet_service.cc:1467] Processing CreateTablet for tablet ecbf44d92c304f30950621004d1a78ab (DEFAULT_TABLE table=test-table [id=6af5fac81e134e28b961bef74371b853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:51.723876 28585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecbf44d92c304f30950621004d1a78ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:51.726387 28776 tablet_replica.cc:330] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: stopping tablet replica
I20260808 02:43:51.726631 28774 tablet_replica.cc:330] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: stopping tablet replica
I20260808 02:43:51.726667 28136 tablet_service.cc:1467] Processing CreateTablet for tablet ecbf44d92c304f30950621004d1a78ab (DEFAULT_TABLE table=test-table [id=6af5fac81e134e28b961bef74371b853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:51.727053 28776 raft_consensus.cc:2229] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:43:51.727159 28136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecbf44d92c304f30950621004d1a78ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:51.728343 28776 raft_consensus.cc:2258] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:43:51.728889 28774 raft_consensus.cc:2229] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:43:51.729485 28774 raft_consensus.cc:2258] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:43:51.731885 28776 ts_tablet_manager.cc:1905] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:43:51.733582 28738 tablet_bootstrap.cc:492] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5: Bootstrap starting.
I20260808 02:43:51.734125 28371 tablet_service.cc:1467] Processing CreateTablet for tablet ecbf44d92c304f30950621004d1a78ab (DEFAULT_TABLE table=test-table [id=6af5fac81e134e28b961bef74371b853]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:51.734560 28371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecbf44d92c304f30950621004d1a78ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:51.735190 28738 tablet_bootstrap.cc:654] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:51.739099 28732 tablet_bootstrap.cc:492] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770: Bootstrap starting.
I20260808 02:43:51.740836 28732 tablet_bootstrap.cc:654] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:51.741495 28778 tablet_replica.cc:330] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: stopping tablet replica
I20260808 02:43:51.741945 28778 raft_consensus.cc:2229] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:43:51.742761 28776 ts_tablet_manager.cc:1918] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:43:51.742884 28778 raft_consensus.cc:2258] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:43:51.743088 28776 log.cc:1196] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/ddee17b7b9234f0ca957e775e7e1b631
I20260808 02:43:51.743741 28776 ts_tablet_manager.cc:1939] T ddee17b7b9234f0ca957e775e7e1b631 P a48476a3e6a4470d9f0c49c3c2948a13: Deleting consensus metadata
I20260808 02:43:51.745153 28778 ts_tablet_manager.cc:1905] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:43:51.745549 28774 ts_tablet_manager.cc:1905] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:43:51.746032 27910 catalog_manager.cc:4841] TS a48476a3e6a4470d9f0c49c3c2948a13 (127.25.137.194:39823): tablet ddee17b7b9234f0ca957e775e7e1b631 (table test-table [id=6bf3488c842540378efafcbd916ec301]) successfully deleted
I20260808 02:43:51.745831 28732 tablet_bootstrap.cc:492] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770: No bootstrap required, opened a new log
I20260808 02:43:51.746399 28732 ts_tablet_manager.cc:1397] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260808 02:43:51.747267 28732 raft_consensus.cc:348] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.747556 28738 tablet_bootstrap.cc:492] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5: No bootstrap required, opened a new log
I20260808 02:43:51.747557 28732 raft_consensus.cc:374] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:51.747663 28732 raft_consensus.cc:729] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29026c9259604618be10e58225dae770, State: Initialized, Role: FOLLOWER
I20260808 02:43:51.747761 28738 ts_tablet_manager.cc:1397] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260808 02:43:51.748019 28732 consensus_queue.cc:260] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.748596 28738 raft_consensus.cc:348] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.748909 28738 raft_consensus.cc:374] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:51.749089 28738 raft_consensus.cc:729] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d74c26a4af454189d5f9a2039a52a5, State: Initialized, Role: FOLLOWER
I20260808 02:43:51.749459 28738 consensus_queue.cc:260] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.749655 28734 tablet_bootstrap.cc:492] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13: Bootstrap starting.
I20260808 02:43:51.750945 28732 ts_tablet_manager.cc:1428] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260808 02:43:51.751251 28734 tablet_bootstrap.cc:654] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:51.752166 28738 ts_tablet_manager.cc:1428] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260808 02:43:51.752995 28778 ts_tablet_manager.cc:1918] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:43:51.753181 28778 log.cc:1196] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/ddee17b7b9234f0ca957e775e7e1b631
I20260808 02:43:51.756012 28734 tablet_bootstrap.cc:492] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13: No bootstrap required, opened a new log
I20260808 02:43:51.756136 28734 ts_tablet_manager.cc:1397] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:43:51.756244 28778 ts_tablet_manager.cc:1939] T ddee17b7b9234f0ca957e775e7e1b631 P d7d74c26a4af454189d5f9a2039a52a5: Deleting consensus metadata
I20260808 02:43:51.757036 28734 raft_consensus.cc:348] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.757413 28734 raft_consensus.cc:374] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:51.757556 28734 raft_consensus.cc:729] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48476a3e6a4470d9f0c49c3c2948a13, State: Initialized, Role: FOLLOWER
I20260808 02:43:51.757920 27910 catalog_manager.cc:4841] TS d7d74c26a4af454189d5f9a2039a52a5 (127.25.137.193:33139): tablet ddee17b7b9234f0ca957e775e7e1b631 (table test-table [id=6bf3488c842540378efafcbd916ec301]) successfully deleted
I20260808 02:43:51.757913 28734 consensus_queue.cc:260] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.758489 28734 ts_tablet_manager.cc:1428] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260808 02:43:51.759819 28774 ts_tablet_manager.cc:1918] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:43:51.760046 28774 log.cc:1196] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/ddee17b7b9234f0ca957e775e7e1b631
I20260808 02:43:51.760504 28774 ts_tablet_manager.cc:1939] T ddee17b7b9234f0ca957e775e7e1b631 P 29026c9259604618be10e58225dae770: Deleting consensus metadata
I20260808 02:43:51.762207 27912 catalog_manager.cc:4841] TS 29026c9259604618be10e58225dae770 (127.25.137.195:44121): tablet ddee17b7b9234f0ca957e775e7e1b631 (table test-table [id=6bf3488c842540378efafcbd916ec301]) successfully deleted
I20260808 02:43:51.874461 28739 raft_consensus.cc:482] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:43:51.874737 28739 raft_consensus.cc:504] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.876562 28739 leader_election.cc:288] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7d74c26a4af454189d5f9a2039a52a5 (127.25.137.193:33139), a48476a3e6a4470d9f0c49c3c2948a13 (127.25.137.194:39823)
I20260808 02:43:51.883401 28785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.876722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:33139 (local address 127.25.137.195:39087)
0808 02:43:51.877346 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:51.877365 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:51.877549 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:51.878128 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:51.878132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:51.878154 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:51.878454 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.878467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.879941 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.879946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.881473 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.881487 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.882613 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.882620 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.882813 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.882850 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:43:51.882946 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 02:43:51.883085 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":363,"threads_started":1}
I20260808 02:43:51.883448 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.876879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:39087 (local address 127.25.137.193:33139)
0808 02:43:51.877108 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.877115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.877437 (+ 322us) server_negotiation.cc:407] Connection header received
0808 02:43:51.877738 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.877745 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.877836 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.877905 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:51.878636 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.879785 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.881624 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.882429 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.882584 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.882660 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.882995 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.883131 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:43:51.884629 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.877697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:42997 (local address 127.25.137.194:39823)
0808 02:43:51.877874 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.877880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.878308 (+ 428us) server_negotiation.cc:407] Connection header received
0808 02:43:51.878484 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.878489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.878565 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.878642 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:51.879201 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.880009 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.882480 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.883506 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.883660 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.883727 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.884227 (+ 500us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.884368 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:43:51.884627 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.877504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:39823 (local address 127.25.137.195:42997)
0808 02:43:51.878174 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:51.878193 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:51.878350 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:51.878767 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:51.878770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:51.878789 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:51.879062 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.879070 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.880152 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.880157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.882302 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.882319 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.883908 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.883917 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.884072 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.884095 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:43:51.884175 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:43:51.884297 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":374,"threads_started":1}
I20260808 02:43:51.884577 28169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecbf44d92c304f30950621004d1a78ab" candidate_uuid: "29026c9259604618be10e58225dae770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d74c26a4af454189d5f9a2039a52a5" is_pre_election: true
I20260808 02:43:51.885119 28169 raft_consensus.cc:2454] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29026c9259604618be10e58225dae770 in term 0.
I20260808 02:43:51.885900 28515 leader_election.cc:302] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29026c9259604618be10e58225dae770, d7d74c26a4af454189d5f9a2039a52a5; no voters:
I20260808 02:43:51.886215 28404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecbf44d92c304f30950621004d1a78ab" candidate_uuid: "29026c9259604618be10e58225dae770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" is_pre_election: true
I20260808 02:43:51.886325 28739 raft_consensus.cc:2781] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:43:51.886435 28739 raft_consensus.cc:482] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:43:51.886474 28404 raft_consensus.cc:2454] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29026c9259604618be10e58225dae770 in term 0.
I20260808 02:43:51.886512 28739 raft_consensus.cc:3037] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:51.889869 28739 raft_consensus.cc:504] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.890583 28739 leader_election.cc:288] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [CANDIDATE]: Term 1 election: Requested vote from peers d7d74c26a4af454189d5f9a2039a52a5 (127.25.137.193:33139), a48476a3e6a4470d9f0c49c3c2948a13 (127.25.137.194:39823)
I20260808 02:43:51.891584 28404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecbf44d92c304f30950621004d1a78ab" candidate_uuid: "29026c9259604618be10e58225dae770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48476a3e6a4470d9f0c49c3c2948a13"
I20260808 02:43:51.891824 28404 raft_consensus.cc:3037] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:51.895565 28169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecbf44d92c304f30950621004d1a78ab" candidate_uuid: "29026c9259604618be10e58225dae770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d74c26a4af454189d5f9a2039a52a5"
I20260808 02:43:51.895829 28169 raft_consensus.cc:3037] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:51.897594 28404 raft_consensus.cc:2454] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29026c9259604618be10e58225dae770 in term 1.
I20260808 02:43:51.899371 28169 raft_consensus.cc:2454] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29026c9259604618be10e58225dae770 in term 1.
I20260808 02:43:51.900115 28515 leader_election.cc:302] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29026c9259604618be10e58225dae770, a48476a3e6a4470d9f0c49c3c2948a13; no voters:
I20260808 02:43:51.901235 28739 raft_consensus.cc:2781] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:43:51.901724 28739 raft_consensus.cc:686] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 1 LEADER]: Becoming Leader. State: Replica: 29026c9259604618be10e58225dae770, State: Running, Role: LEADER
I20260808 02:43:51.902218 28739 consensus_queue.cc:237] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.905893 27928 catalog_manager.cc:5495] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29026c9259604618be10e58225dae770 (127.25.137.195). New cstate: current_term: 1 leader_uuid: "29026c9259604618be10e58225dae770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } health_report { overall_health: HEALTHY } } }
Aug 08 02:43:51 dist-test-slave-k5rr krb5kdc[27867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157029, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:51.969245 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.955241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:33315 (local address 127.0.0.1:38086)
0808 02:43:51.955926 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:51.955948 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:51.956120 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:51.956518 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:51.956523 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:51.957296 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:51.957568 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.957578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.958673 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.958677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.959660 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:51.959674 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.960387 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.960397 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:51.960597 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.961294 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:51.961316 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:51.963693 (+ 2377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:51.966550 (+ 2857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:51.966562 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:51.966570 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:51.966967 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:51.967339 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:51.967344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:51.967348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:51.967487 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:51.968045 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:51.968052 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:51.968287 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 02:43:51.968451 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 02:43:51.968717 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":326,"threads_started":1}
I20260808 02:43:51.969290 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:51.955326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38086 (local address 127.25.137.254:33315)
0808 02:43:51.956251 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:51.956255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:51.956271 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:43:51.956323 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:51.956327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:51.956380 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:51.956466 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:51.957728 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.958542 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.959800 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:51.960205 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:51.960324 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:51.963899 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:51.963903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:51.963906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:51.963934 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:51.966433 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:51.967073 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:51.967078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:51.967100 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:51.967234 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:51.967586 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:51.967591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:51.967595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:51.967928 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:51.968054 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:51.968545 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 02:43:51.968878 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":809,"thread_start_us":669,"threads_started":1}
I20260808 02:43:51.975431 27928 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:38086:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:43:51.976699 27928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:43:51.983836 28137 tablet_service.cc:1467] Processing CreateTablet for tablet ea15c9afa4bc4d84974ba8dc9c2644c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d86a7b08c7bf43ce8ab9ea8e4416777c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:43:51.984071 28371 tablet_service.cc:1467] Processing CreateTablet for tablet ea15c9afa4bc4d84974ba8dc9c2644c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d86a7b08c7bf43ce8ab9ea8e4416777c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:43:51.984450 28137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea15c9afa4bc4d84974ba8dc9c2644c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:51.984387 28585 tablet_service.cc:1467] Processing CreateTablet for tablet ea15c9afa4bc4d84974ba8dc9c2644c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d86a7b08c7bf43ce8ab9ea8e4416777c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:43:51.984642 28371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea15c9afa4bc4d84974ba8dc9c2644c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:51.984951 28585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea15c9afa4bc4d84974ba8dc9c2644c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:51.990238 28734 tablet_bootstrap.cc:492] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13: Bootstrap starting.
I20260808 02:43:51.991971 28738 tablet_bootstrap.cc:492] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5: Bootstrap starting.
I20260808 02:43:51.991992 28734 tablet_bootstrap.cc:654] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:51.993965 28738 tablet_bootstrap.cc:654] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:51.994495 28734 tablet_bootstrap.cc:492] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13: No bootstrap required, opened a new log
I20260808 02:43:51.994652 28734 ts_tablet_manager.cc:1397] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260808 02:43:51.995635 28734 raft_consensus.cc:348] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.995899 28734 raft_consensus.cc:374] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:51.995991 28734 raft_consensus.cc:729] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48476a3e6a4470d9f0c49c3c2948a13, State: Initialized, Role: FOLLOWER
I20260808 02:43:51.996485 28738 tablet_bootstrap.cc:492] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5: No bootstrap required, opened a new log
I20260808 02:43:51.996308 28734 consensus_queue.cc:260] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.996661 28738 ts_tablet_manager.cc:1397] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 02:43:51.996980 28734 ts_tablet_manager.cc:1428] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:43:51.997730 28738 raft_consensus.cc:348] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.998083 28738 raft_consensus.cc:374] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:51.997191 28741 tablet_replica.cc:439] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } } }
I20260808 02:43:51.998252 28738 raft_consensus.cc:729] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d74c26a4af454189d5f9a2039a52a5, State: Initialized, Role: FOLLOWER
I20260808 02:43:51.998358 28741 tablet_replica.cc:442] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:51.998540 28738 consensus_queue.cc:260] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:51.998939 28732 tablet_bootstrap.cc:492] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: Bootstrap starting.
I20260808 02:43:51.999061 28766 tablet_replica.cc:439] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } } }
I20260808 02:43:51.999432 28766 tablet_replica.cc:442] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:52.000576 28738 ts_tablet_manager.cc:1428] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260808 02:43:52.000792 28732 tablet_bootstrap.cc:654] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:52.004290 28732 tablet_bootstrap.cc:492] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: No bootstrap required, opened a new log
I20260808 02:43:52.004454 28732 ts_tablet_manager.cc:1397] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20260808 02:43:52.005189 28732 raft_consensus.cc:348] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:52.005406 28732 raft_consensus.cc:374] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:52.005483 28732 raft_consensus.cc:729] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29026c9259604618be10e58225dae770, State: Initialized, Role: FOLLOWER
I20260808 02:43:52.005709 28732 consensus_queue.cc:260] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:52.006346 28732 ts_tablet_manager.cc:1428] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260808 02:43:52.006191 28739 tablet_replica.cc:439] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } } }
I20260808 02:43:52.006938 28739 tablet_replica.cc:442] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:52.286440 28739 consensus_queue.cc:1035] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:43:52.293336 28812 consensus_queue.cc:1035] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:43:52.321429 28812 raft_consensus.cc:482] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:43:52.321642 28812 raft_consensus.cc:504] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:52.322288 28812 leader_election.cc:288] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7d74c26a4af454189d5f9a2039a52a5 (127.25.137.193:33139), a48476a3e6a4470d9f0c49c3c2948a13 (127.25.137.194:39823)
I20260808 02:43:52.322861 28169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea15c9afa4bc4d84974ba8dc9c2644c8" candidate_uuid: "29026c9259604618be10e58225dae770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d74c26a4af454189d5f9a2039a52a5" is_pre_election: true
I20260808 02:43:52.322906 28404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea15c9afa4bc4d84974ba8dc9c2644c8" candidate_uuid: "29026c9259604618be10e58225dae770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" is_pre_election: true
I20260808 02:43:52.323082 28169 raft_consensus.cc:2454] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29026c9259604618be10e58225dae770 in term 0.
I20260808 02:43:52.323154 28404 raft_consensus.cc:2454] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29026c9259604618be10e58225dae770 in term 0.
I20260808 02:43:52.323527 28515 leader_election.cc:302] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29026c9259604618be10e58225dae770, d7d74c26a4af454189d5f9a2039a52a5; no voters:
I20260808 02:43:52.323817 28812 raft_consensus.cc:2781] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:43:52.323916 28812 raft_consensus.cc:482] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:43:52.324023 28812 raft_consensus.cc:3037] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:52.326406 28812 raft_consensus.cc:504] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:52.326910 28812 leader_election.cc:288] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [CANDIDATE]: Term 1 election: Requested vote from peers d7d74c26a4af454189d5f9a2039a52a5 (127.25.137.193:33139), a48476a3e6a4470d9f0c49c3c2948a13 (127.25.137.194:39823)
I20260808 02:43:52.327350 28169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea15c9afa4bc4d84974ba8dc9c2644c8" candidate_uuid: "29026c9259604618be10e58225dae770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d74c26a4af454189d5f9a2039a52a5"
I20260808 02:43:52.327412 28404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea15c9afa4bc4d84974ba8dc9c2644c8" candidate_uuid: "29026c9259604618be10e58225dae770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48476a3e6a4470d9f0c49c3c2948a13"
I20260808 02:43:52.327545 28169 raft_consensus.cc:3037] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:52.327628 28404 raft_consensus.cc:3037] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:52.329587 28169 raft_consensus.cc:2454] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29026c9259604618be10e58225dae770 in term 1.
I20260808 02:43:52.329658 28404 raft_consensus.cc:2454] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29026c9259604618be10e58225dae770 in term 1.
I20260808 02:43:52.329967 28515 leader_election.cc:302] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29026c9259604618be10e58225dae770, d7d74c26a4af454189d5f9a2039a52a5; no voters:
I20260808 02:43:52.330233 28812 raft_consensus.cc:2781] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:43:52.330412 28812 raft_consensus.cc:686] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [term 1 LEADER]: Becoming Leader. State: Replica: 29026c9259604618be10e58225dae770, State: Running, Role: LEADER
I20260808 02:43:52.330662 28812 consensus_queue.cc:237] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } }
I20260808 02:43:52.332271 28789 tablet_replica.cc:439] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: TxnStatusTablet state changed. Reason: New leader 29026c9259604618be10e58225dae770. Latest consensus state: current_term: 1 leader_uuid: "29026c9259604618be10e58225dae770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } } }
I20260808 02:43:52.332579 28789 tablet_replica.cc:442] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:43:52.333148 28819 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:43:52.333487 28819 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:43:52.333655 27928 catalog_manager.cc:5495] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29026c9259604618be10e58225dae770 (127.25.137.195). New cstate: current_term: 1 leader_uuid: "29026c9259604618be10e58225dae770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } health_report { overall_health: HEALTHY } } }
I20260808 02:43:52.377475 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:52.370758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56598 (local address 127.25.137.195:44121)
0808 02:43:52.371297 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:52.371301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:52.371327 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:43:52.371463 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:52.371467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:52.371596 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:52.371754 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:43:52.372202 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.373122 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.375132 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.375616 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.375713 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:52.376249 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:43:52.376836 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:43:52.376943 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:52.377097 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 02:43:52.377165 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":305,"threads_started":1}
I20260808 02:43:52.377609 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:52.370710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:44121 (local address 127.0.0.1:56598)
0808 02:43:52.371079 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:52.371101 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:52.371242 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:52.371776 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:52.371782 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:52.371796 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:43:52.372016 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:52.372029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.373253 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.373259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:52.374979 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:52.374998 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.375778 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.375784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:52.375991 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:52.376072 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:43:52.376946 (+ 874us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:43:52.376968 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:43:52.377080 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 02:43:52.377285 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260808 02:43:52.381989 28404 raft_consensus.cc:1261] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13 [term 1 FOLLOWER]: Refusing update from remote peer 29026c9259604618be10e58225dae770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:43:52.382093 28169 raft_consensus.cc:1261] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5 [term 1 FOLLOWER]: Refusing update from remote peer 29026c9259604618be10e58225dae770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:43:52.382670 28812 consensus_queue.cc:1035] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:43:52.382990 28789 consensus_queue.cc:1035] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:43:52.408361 28766 tablet_replica.cc:439] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5: TxnStatusTablet state changed. Reason: New leader 29026c9259604618be10e58225dae770. Latest consensus state: current_term: 1 leader_uuid: "29026c9259604618be10e58225dae770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } } }
I20260808 02:43:52.408680 28766 tablet_replica.cc:442] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:52.409847 28741 tablet_replica.cc:439] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13: TxnStatusTablet state changed. Reason: New leader 29026c9259604618be10e58225dae770. Latest consensus state: current_term: 1 leader_uuid: "29026c9259604618be10e58225dae770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } } }
I20260808 02:43:52.410111 28741 tablet_replica.cc:442] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:52.411415 28789 tablet_replica.cc:439] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29026c9259604618be10e58225dae770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } } }
I20260808 02:43:52.411749 28789 tablet_replica.cc:442] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:43:52.413676 28813 mvcc.cc:205] Tried to move back new op lower bound from 7316099204630134784 to 7316099204428353536. Current Snapshot: MvccSnapshot[applied={T|T < 7316099204630134784}]
I20260808 02:43:52.419718 28789 tablet_replica.cc:439] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29026c9259604618be10e58225dae770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } } }
I20260808 02:43:52.419992 28789 tablet_replica.cc:442] T ea15c9afa4bc4d84974ba8dc9c2644c8 P 29026c9259604618be10e58225dae770: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:43:52.431756 28766 tablet_replica.cc:439] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29026c9259604618be10e58225dae770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } } }
I20260808 02:43:52.432067 28766 tablet_replica.cc:442] T ea15c9afa4bc4d84974ba8dc9c2644c8 P d7d74c26a4af454189d5f9a2039a52a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:52.441293 28741 tablet_replica.cc:439] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29026c9259604618be10e58225dae770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d74c26a4af454189d5f9a2039a52a5" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33139 } } peers { permanent_uuid: "a48476a3e6a4470d9f0c49c3c2948a13" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39823 } } peers { permanent_uuid: "29026c9259604618be10e58225dae770" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 44121 } } }
I20260808 02:43:52.441562 28741 tablet_replica.cc:442] T ea15c9afa4bc4d84974ba8dc9c2644c8 P a48476a3e6a4470d9f0c49c3c2948a13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:52.448478 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:52.430808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56604 (local address 127.25.137.195:44121)
0808 02:43:52.431460 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:52.431466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:52.441867 (+ 10401us) server_negotiation.cc:407] Connection header received
0808 02:43:52.442066 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:52.442072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:52.442159 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:52.442259 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:43:52.443132 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.444396 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.446503 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.447219 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.447349 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:52.447834 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:43:52.447953 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:43:52.448027 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:52.448159 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 02:43:52.448230 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526}
I20260808 02:43:52.465256 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:52.459642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:55905 (local address 127.25.137.195:44121)
0808 02:43:52.459793 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:52.459797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:52.460411 (+ 614us) server_negotiation.cc:407] Connection header received
0808 02:43:52.460611 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:52.460615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:52.460678 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:52.460749 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:43:52.461440 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.462193 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.463566 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.464004 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.464111 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:52.464524 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:43:52.464625 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:43:52.464694 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:52.464821 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 02:43:52.464892 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:43:52.465332 28843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:52.459702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:44121 (local address 127.25.137.195:55905)
0808 02:43:52.460318 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:52.460333 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:52.460472 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:52.460776 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:52.460780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:52.460791 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:43:52.461245 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:52.461258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.462323 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.462327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:52.463430 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:52.463442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.464172 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.464178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:52.464294 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:52.464350 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:43:52.464713 (+ 363us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:43:52.464723 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 02:43:52.464790 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 02:43:52.464926 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":299,"threads_started":1}
I20260808 02:43:52.534963 27928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:43:52.535197 27928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:43:52.538270 27928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd9696fe4a7a480f855fa25f8dc34ddb: Sending DeleteTablet for 3 replicas of tablet ecbf44d92c304f30950621004d1a78ab
I20260808 02:43:52.539103 28371 tablet_service.cc:1514] Processing DeleteTablet for tablet ecbf44d92c304f30950621004d1a78ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:43:52 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:43800
I20260808 02:43:52.539093 28137 tablet_service.cc:1514] Processing DeleteTablet for tablet ecbf44d92c304f30950621004d1a78ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:43:52 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:41728
I20260808 02:43:52.539534 28586 tablet_service.cc:1514] Processing DeleteTablet for tablet ecbf44d92c304f30950621004d1a78ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:43:52 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:38480
I20260808 02:43:52.555605 28848 tablet_replica.cc:330] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770: stopping tablet replica
I20260808 02:43:52.555842 28848 raft_consensus.cc:2229] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:43:52.556458 28848 raft_consensus.cc:2258] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:43:52.557974 28848 ts_tablet_manager.cc:1905] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:43:52.561180 28845 tablet_replica.cc:330] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13: stopping tablet replica
I20260808 02:43:52.561520 28845 raft_consensus.cc:2229] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:43:52.561834 28845 raft_consensus.cc:2258] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:43:52.562413 28846 tablet_replica.cc:330] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5: stopping tablet replica
I20260808 02:43:52.562597 28846 raft_consensus.cc:2229] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:43:52.562832 28846 raft_consensus.cc:2258] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:43:52.563124 28845 ts_tablet_manager.cc:1905] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:43:52.564023 28846 ts_tablet_manager.cc:1905] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:43:52.565532 28848 ts_tablet_manager.cc:1918] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:43:52.565671 28848 log.cc:1196] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/ecbf44d92c304f30950621004d1a78ab
I20260808 02:43:52.566141 28848 ts_tablet_manager.cc:1939] T ecbf44d92c304f30950621004d1a78ab P 29026c9259604618be10e58225dae770: Deleting consensus metadata
Aug 08 02:43:52 dist-test-slave-k5rr krb5kdc[27867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157031, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:43:52.567273 27912 catalog_manager.cc:4841] TS 29026c9259604618be10e58225dae770 (127.25.137.195:44121): tablet ecbf44d92c304f30950621004d1a78ab (table test-table [id=6af5fac81e134e28b961bef74371b853]) successfully deleted
I20260808 02:43:52.570364 28845 ts_tablet_manager.cc:1918] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:43:52.570546 28845 log.cc:1196] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/ecbf44d92c304f30950621004d1a78ab
I20260808 02:43:52.571038 28845 ts_tablet_manager.cc:1939] T ecbf44d92c304f30950621004d1a78ab P a48476a3e6a4470d9f0c49c3c2948a13: Deleting consensus metadata
I20260808 02:43:52.572263 27910 catalog_manager.cc:4841] TS a48476a3e6a4470d9f0c49c3c2948a13 (127.25.137.194:39823): tablet ecbf44d92c304f30950621004d1a78ab (table test-table [id=6af5fac81e134e28b961bef74371b853]) successfully deleted
I20260808 02:43:52.572454 28846 ts_tablet_manager.cc:1918] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:43:52.572577 28846 log.cc:1196] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/ecbf44d92c304f30950621004d1a78ab
I20260808 02:43:52.573050 28846 ts_tablet_manager.cc:1939] T ecbf44d92c304f30950621004d1a78ab P d7d74c26a4af454189d5f9a2039a52a5: Deleting consensus metadata
I20260808 02:43:52.574163 27910 catalog_manager.cc:4841] TS d7d74c26a4af454189d5f9a2039a52a5 (127.25.137.193:33139): tablet ecbf44d92c304f30950621004d1a78ab (table test-table [id=6af5fac81e134e28b961bef74371b853]) successfully deleted
I20260808 02:43:52.576098 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:52.550571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59850 (local address 127.25.137.193:33139)
0808 02:43:52.551148 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:52.551154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:52.552185 (+ 1031us) server_negotiation.cc:407] Connection header received
0808 02:43:52.552510 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:52.552517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:52.552608 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:52.552689 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:52.555377 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.556462 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.560384 (+ 3922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.561089 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.561187 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:52.567885 (+ 6698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:52.567900 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:52.567918 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:52.567969 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:52.571961 (+ 3992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:52.572894 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:52.572907 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:52.572911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:52.573148 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:52.573669 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:52.573674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:52.573678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:52.574203 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:52.574524 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:52.575249 (+ 725us) server_negotiation.cc:299] Negotiation successful
0808 02:43:52.575630 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":269,"threads_started":1}
W20260808 02:43:52.578516 28248 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59850
I20260808 02:43:52.600895 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:52.584394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.25.137.254:33315)
0808 02:43:52.584939 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:52.584946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:52.585468 (+ 522us) server_negotiation.cc:407] Connection header received
0808 02:43:52.585664 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:52.585670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:52.585760 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:52.585888 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:52.587355 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.588499 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.590398 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.591060 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.591185 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:52.594166 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:52.594173 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:52.594178 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:52.594221 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:52.597900 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:52.598795 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:52.598801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:52.598805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:52.598955 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:52.599423 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:52.599427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:52.599429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:52.599700 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:52.599963 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:52.600324 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 02:43:52.600553 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":231,"threads_started":1}
W20260808 02:43:52.601863 27928 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38100
I20260808 02:43:52.605742 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28035
I20260808 02:43:52.622576 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:52.607333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59860 (local address 127.25.137.193:33139)
0808 02:43:52.607579 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:52.607586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:52.608392 (+ 806us) server_negotiation.cc:407] Connection header received
0808 02:43:52.608586 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:52.608592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:52.608729 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:52.608859 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:52.610421 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.611552 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.613416 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.614074 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.614207 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:52.616638 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:52.616645 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:52.616650 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:52.616693 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:52.619986 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:52.620649 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:52.620655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:52.620658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:52.620809 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:52.621217 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:52.621222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:52.621232 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:52.621534 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:52.621764 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:52.622006 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 02:43:52.622233 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260808 02:43:52.623126 28248 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59860
W20260808 02:43:52.623641 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28035: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:43:52.623821 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 28035
I20260808 02:43:52.655776 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28263
Aug 08 02:43:52 dist-test-slave-k5rr krb5kdc[27867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157031, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:43:52.671427 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:52.656613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39794 (local address 127.25.137.194:39823)
0808 02:43:52.657229 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:52.657234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:52.657250 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:43:52.657310 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:52.657314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:52.657373 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:52.657476 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:52.658627 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.659369 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.660625 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.661122 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.661236 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:52.665512 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:52.665527 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:52.665543 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:52.665578 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:52.668514 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:52.669301 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:52.669309 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:52.669311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:52.669441 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:52.669796 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:52.669801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:52.669804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:52.670163 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:52.670452 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:52.670767 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 02:43:52.670990 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":312,"threads_started":1}
W20260808 02:43:52.672390 28474 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39794
W20260808 02:43:52.673054 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28263: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:43:52.673245 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 28263
I20260808 02:43:52.702298 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28487
Aug 08 02:43:52 dist-test-slave-k5rr krb5kdc[27867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157031, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:43:52.720741 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:52.703068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.25.137.195:44121)
0808 02:43:52.703263 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:52.703268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:52.703287 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:43:52.703472 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:52.703477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:52.703537 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:52.703615 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:52.705457 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.706500 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.708400 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:52.708950 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:52.709220 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:52.714639 (+ 5419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:52.714658 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:52.714674 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:52.714712 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:52.717891 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:52.718745 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:52.718752 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:52.718754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:52.718886 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:52.719294 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:52.719297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:52.719299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:52.719661 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:52.719904 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:52.720270 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 02:43:52.720447 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 02:43:52.721556 28687 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56610
W20260808 02:43:52.722262 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28487: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:43:52.722458 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 28487
I20260808 02:43:52.750619 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27884
I20260808 02:43:52.751746 27984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:43:52.867408 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 27884
2026-08-08T02:43:52Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4879 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:43:52 dist-test-slave-k5rr krb5kdc[28865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:43:52 dist-test-slave-k5rr krb5kdc[28865](info): set up 2 sockets
Aug 08 02:43:52 dist-test-slave-k5rr krb5kdc[28865](info): commencing operation
krb5kdc: starting...
W20260808 02:43:54.152840 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260808 02:43:54.274030 26151 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.121s user 0.000s sys 0.007s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260808 02:43:54.413647 26151 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.139s user 0.000s sys 0.008s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260808 02:43:54.547173 26151 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.133s user 0.002s sys 0.006s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/test-user.keytab.
W20260808 02:43:55.249589 26151 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.702s user 0.000s sys 0.007s
Aug 08 02:43:55 dist-test-slave-k5rr krb5kdc[28865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:43:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:43:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
W20260808 02:43:55.597067 26151 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.25.137.254: real 0.289s user 0.003s sys 0.010s
I20260808 02:43:55.598153 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:42093
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:42093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:55.917533 28884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:55.917835 28884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:55.917971 28884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:55.925068 28884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:43:55.925179 28884 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:43:55.925248 28884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:55.925341 28884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:55.925411 28884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:43:55.925493 28884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:43:55.935082 28884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:42093
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:42093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:55.936398 28884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:55.937762 28884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:55.948901 28895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:55.948884 28892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:55.948851 28889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:55.950053 28884 server_base.cc:1034] running on GCE node
I20260808 02:43:55.950865 28884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:55.952317 28884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:55.953529 28884 hybrid_clock.cc:648] HybridClock initialized: now 1786157035953453 us; error 50 us; skew 500 ppm
I20260808 02:43:55.953855 28884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:55 dist-test-slave-k5rr krb5kdc[28865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157035, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:55.960842 28884 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:43:55.962378 28884 webserver.cc:460] Webserver started at http://127.25.137.254:46253/ using document root <none> and password file <none>
I20260808 02:43:55.963235 28884 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:55.963374 28884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:55.963702 28884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:55.966797 28884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "6c2c437417614743865926995c658d70"
format_stamp: "Formatted at 2026-08-08 02:43:55 on dist-test-slave-k5rr"
server_key: "72592967fcc1ed2808deaf80ee5cdbc6"
server_key_iv: "248242b1d15f156583b35f36cabf350e"
server_key_version: "encryptionkey@0"
I20260808 02:43:55.967603 28884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "6c2c437417614743865926995c658d70"
format_stamp: "Formatted at 2026-08-08 02:43:55 on dist-test-slave-k5rr"
server_key: "72592967fcc1ed2808deaf80ee5cdbc6"
server_key_iv: "248242b1d15f156583b35f36cabf350e"
server_key_version: "encryptionkey@0"
I20260808 02:43:55.972749 28884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:43:55.976235 28908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:55.977591 28884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:43:55.977753 28884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "6c2c437417614743865926995c658d70"
format_stamp: "Formatted at 2026-08-08 02:43:55 on dist-test-slave-k5rr"
server_key: "72592967fcc1ed2808deaf80ee5cdbc6"
server_key_iv: "248242b1d15f156583b35f36cabf350e"
server_key_version: "encryptionkey@0"
I20260808 02:43:55.977970 28884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:56.013708 28884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:56.018157 28884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:56.018456 28884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:56.040632 28884 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:42093
I20260808 02:43:56.040661 29000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:42093 every 8 connection(s)
I20260808 02:43:56.042129 28884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:43:56.045866 29001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:56.048862 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 28884
I20260808 02:43:56.049110 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:43:56.049806 26151 external_mini_cluster.cc:1428] Setting key 5873034dd6ebc70222f485aac476f1ec
I20260808 02:43:56.058429 29001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70: Bootstrap starting.
I20260808 02:43:56.062126 29001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:56.063412 29001 log.cc:824] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:43:56 dist-test-slave-k5rr krb5kdc[28865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:56.066437 29001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70: No bootstrap required, opened a new log
I20260808 02:43:56.073925 29004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:56.051831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35864 (local address 127.25.137.254:42093)
0808 02:43:56.052596 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:56.052608 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:56.052651 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:43:56.054426 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:56.054455 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:56.054862 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:56.055206 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:56.057046 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:56.058382 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:56.060091 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:56.060837 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:56.060983 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:56.065580 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:56.065592 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:56.065611 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:56.065656 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:56.069571 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:56.070184 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:56.070193 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:56.070202 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:56.070403 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:56.070747 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:56.070752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:56.070756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:56.071416 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:56.071740 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:56.072087 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 02:43:56.072512 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":331,"threads_started":1}
I20260808 02:43:56.076681 29001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2c437417614743865926995c658d70" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42093 } }
I20260808 02:43:56.077136 29001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:56.077236 29001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c2c437417614743865926995c658d70, State: Initialized, Role: FOLLOWER
I20260808 02:43:56.077776 29001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2c437417614743865926995c658d70" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42093 } }
I20260808 02:43:56.077989 29001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:43:56.078111 29001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:43:56.078269 29001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:56.080412 29001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2c437417614743865926995c658d70" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42093 } }
I20260808 02:43:56.080883 29001 leader_election.cc:302] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c2c437417614743865926995c658d70; no voters:
I20260808 02:43:56.081591 29001 leader_election.cc:288] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:43:56.081696 29007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:43:56.082860 29007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [term 1 LEADER]: Becoming Leader. State: Replica: 6c2c437417614743865926995c658d70, State: Running, Role: LEADER
I20260808 02:43:56.083359 29007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2c437417614743865926995c658d70" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42093 } }
I20260808 02:43:56.083760 29001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:43:56.086290 29008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c2c437417614743865926995c658d70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2c437417614743865926995c658d70" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42093 } } }
I20260808 02:43:56.086320 29009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c2c437417614743865926995c658d70. Latest consensus state: current_term: 1 leader_uuid: "6c2c437417614743865926995c658d70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c2c437417614743865926995c658d70" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42093 } } }
I20260808 02:43:56.086701 29009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [sys.catalog]: This master's current role is: LEADER
I20260808 02:43:56.086697 29008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70 [sys.catalog]: This master's current role is: LEADER
I20260808 02:43:56.088650 29017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:43:56.094077 29017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:43:56.102598 29017 catalog_manager.cc:1347] Generated new cluster ID: 09870ea7fbba457f9f771e029fbf17c8
I20260808 02:43:56.102715 29017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:43:56.115797 29017 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:43:56.117146 29017 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:43:56.123101 29017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70: Generated new TSK 0
I20260808 02:43:56.123948 29017 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:43:56.176148 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:42093
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:56.502010 29034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:56.502295 29034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:56.502449 29034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:56.509696 29034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:56.509797 29034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:56.509917 29034 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:43:56.509979 29034 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:43:56.510085 29034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:43:56.519673 29034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.137.254:42093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:56.520949 29034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:56.522300 29034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:56.534729 29042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:56.534801 29043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:56.534691 29049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:56.535953 29034 server_base.cc:1034] running on GCE node
I20260808 02:43:56.536531 29034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:56.537513 29034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:56.538714 29034 hybrid_clock.cc:648] HybridClock initialized: now 1786157036538671 us; error 34 us; skew 500 ppm
I20260808 02:43:56.539005 29034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:56 dist-test-slave-k5rr krb5kdc[28865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157036, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:56.542145 29034 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:43:56.543646 29034 webserver.cc:460] Webserver started at http://127.25.137.193:45935/ using document root <none> and password file <none>
I20260808 02:43:56.544342 29034 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:56.544461 29034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:56.544782 29034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:56.547899 29034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "c3080a7113294155897915ae876400c7"
format_stamp: "Formatted at 2026-08-08 02:43:56 on dist-test-slave-k5rr"
server_key: "a335a80681a7974e8c1ccc598517bca5"
server_key_iv: "711d9705f6a6b8249b613105ae3469f6"
server_key_version: "encryptionkey@0"
I20260808 02:43:56.548698 29034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "c3080a7113294155897915ae876400c7"
format_stamp: "Formatted at 2026-08-08 02:43:56 on dist-test-slave-k5rr"
server_key: "a335a80681a7974e8c1ccc598517bca5"
server_key_iv: "711d9705f6a6b8249b613105ae3469f6"
server_key_version: "encryptionkey@0"
I20260808 02:43:56.554229 29034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:43:56.557727 29058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:56.559026 29034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:43:56.559186 29034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "c3080a7113294155897915ae876400c7"
format_stamp: "Formatted at 2026-08-08 02:43:56 on dist-test-slave-k5rr"
server_key: "a335a80681a7974e8c1ccc598517bca5"
server_key_iv: "711d9705f6a6b8249b613105ae3469f6"
server_key_version: "encryptionkey@0"
I20260808 02:43:56.559391 29034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:56.574759 29034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:56.579166 29034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:56.579466 29034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:56.582479 29034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:56.582613 29034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:56.582754 29034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:56.582854 29034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:43:56 dist-test-slave-k5rr krb5kdc[28865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157036, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:56.606324 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:56.582780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:32783 (local address 127.25.137.254:42093)
0808 02:43:56.585307 (+ 2527us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:56.585312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:56.585330 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:43:56.587588 (+ 2258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:56.587593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:56.587663 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:56.587758 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:56.590142 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:56.591071 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:56.593540 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:56.594080 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:56.594195 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:56.599151 (+ 4956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:56.599157 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:56.599162 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:56.599207 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:56.602117 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:56.603269 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:56.603275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:56.603278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:56.603443 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:56.604070 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:56.604075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:56.604079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:56.604397 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:56.604579 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:56.605667 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0808 02:43:56.605930 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2404,"thread_start_us":2303,"threads_started":1}
I20260808 02:43:56.607880 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:56.583460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42093 (local address 127.25.137.193:32783)
0808 02:43:56.584457 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:56.584518 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:56.587253 (+ 2735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:56.588074 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:56.588086 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:56.588909 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:56.589915 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:56.589949 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:56.591288 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:56.591294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:56.593364 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:56.593388 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:56.594319 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:56.594326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:56.594527 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:56.595362 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:56.595391 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:56.598857 (+ 3466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:56.602302 (+ 3445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:56.602319 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:56.602348 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:56.603097 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:56.603562 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:56.603568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:56.603572 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:56.603900 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:56.604568 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:56.604578 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:56.605127 (+ 549us) client_negotiation.cc:769] Sending connection context
0808 02:43:56.605535 (+ 408us) client_negotiation.cc:240] Negotiation successful
0808 02:43:56.606208 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":383,"threads_started":1}
I20260808 02:43:56.627302 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:56.613420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:36481 (local address 127.25.137.254:42093)
0808 02:43:56.613603 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:56.613609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:56.614301 (+ 692us) server_negotiation.cc:407] Connection header received
0808 02:43:56.614553 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:56.614559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:56.614628 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:56.614711 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:56.616045 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:56.617023 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:56.618461 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:56.619302 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:56.619466 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:56.621907 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:56.621914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:56.621919 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:56.621959 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:56.624699 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:56.625466 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:56.625472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:56.625475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:56.625604 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:56.625932 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:56.625938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:56.625941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:56.626205 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:56.626342 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:56.626726 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 02:43:56.626944 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:43:56.627389 29170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:56.613514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42093 (local address 127.25.137.193:36481)
0808 02:43:56.614167 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:56.614206 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:56.614361 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:56.614772 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:56.614776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:56.615440 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:56.615906 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:56.615920 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:56.617207 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:56.617210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:56.618307 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:56.618320 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:56.619572 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:56.619610 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:56.619906 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:56.620648 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:56.620666 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:56.621640 (+ 974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:56.624828 (+ 3188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:56.624834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:56.624838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:56.625360 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:56.625703 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:56.625708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:56.625713 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:56.625836 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:56.626322 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:56.626327 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:56.626502 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 02:43:56.626620 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 02:43:56.626925 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":338,"threads_started":1}
I20260808 02:43:56.637398 29034 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:41301
I20260808 02:43:56.637420 29234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:41301 every 8 connection(s)
I20260808 02:43:56.638639 29034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:43:56.640640 29235 heartbeater.cc:347] Connected to a master server at 127.25.137.254:42093
I20260808 02:43:56.640856 29235 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:56.641492 29235 heartbeater.cc:511] Master 127.25.137.254:42093 requested a full tablet report, sending...
I20260808 02:43:56.643275 28932 ts_manager.cc:194] Registered new tserver with Master: c3080a7113294155897915ae876400c7 (127.25.137.193:41301)
I20260808 02:43:56.644758 28932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:32783
I20260808 02:43:56.647908 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 29034
I20260808 02:43:56.648108 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:43:56.648720 26151 external_mini_cluster.cc:1428] Setting key 891f822cab8dbd64a636e673af3d968f
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:43:56.692111 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:42093
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:57.010914 29240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:57.011193 29240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:57.011332 29240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:57.018505 29240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:57.018606 29240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:57.018726 29240 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:43:57.018793 29240 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:43:57.018894 29240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:43:57.028364 29240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.137.254:42093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:57.029609 29240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:57.030893 29240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:57.048777 29252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:57.048990 29254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:57.049737 29259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:57.050343 29240 server_base.cc:1034] running on GCE node
I20260808 02:43:57.050988 29240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:57.052080 29240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:57.053295 29240 hybrid_clock.cc:648] HybridClock initialized: now 1786157037053244 us; error 45 us; skew 500 ppm
I20260808 02:43:57.053612 29240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:57 dist-test-slave-k5rr krb5kdc[28865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157037, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:57.056650 29240 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:43:57.058055 29240 webserver.cc:460] Webserver started at http://127.25.137.194:40809/ using document root <none> and password file <none>
I20260808 02:43:57.058712 29240 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:57.058830 29240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:57.059134 29240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:57.062070 29240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "cd859eb96e974fba869481bce2d02bb1"
format_stamp: "Formatted at 2026-08-08 02:43:57 on dist-test-slave-k5rr"
server_key: "2b86142b19d27f941700304857842a66"
server_key_iv: "a529478757e0eb052fa9f8e279b7b9a2"
server_key_version: "encryptionkey@0"
I20260808 02:43:57.062858 29240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "cd859eb96e974fba869481bce2d02bb1"
format_stamp: "Formatted at 2026-08-08 02:43:57 on dist-test-slave-k5rr"
server_key: "2b86142b19d27f941700304857842a66"
server_key_iv: "a529478757e0eb052fa9f8e279b7b9a2"
server_key_version: "encryptionkey@0"
I20260808 02:43:57.067854 29240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:43:57.071223 29268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:57.072532 29240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:43:57.072674 29240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "cd859eb96e974fba869481bce2d02bb1"
format_stamp: "Formatted at 2026-08-08 02:43:57 on dist-test-slave-k5rr"
server_key: "2b86142b19d27f941700304857842a66"
server_key_iv: "a529478757e0eb052fa9f8e279b7b9a2"
server_key_version: "encryptionkey@0"
I20260808 02:43:57.072849 29240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:57.088984 29240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:57.093212 29240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:57.093487 29240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:57.096153 29240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:57.096256 29240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:57.096370 29240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:57.096447 29240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:43:57 dist-test-slave-k5rr krb5kdc[28865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157037, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:57.121948 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.096756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:54631 (local address 127.25.137.254:42093)
0808 02:43:57.096947 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:57.096953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:57.098388 (+ 1435us) server_negotiation.cc:407] Connection header received
0808 02:43:57.101410 (+ 3022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:57.101417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:57.101512 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:57.101637 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:57.104029 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.104978 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.107986 (+ 3008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.108649 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.108821 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.113857 (+ 5036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:57.113862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:57.113866 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:57.113901 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:57.117579 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.118862 (+ 1283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.118869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.118873 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.119053 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.119711 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.119716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.119718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.120167 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:57.120341 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:57.121233 (+ 892us) server_negotiation.cc:299] Negotiation successful
0808 02:43:57.121508 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:43:57.123651 29287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.097354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42093 (local address 127.25.137.194:54631)
0808 02:43:57.098182 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:57.098238 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:57.100949 (+ 2711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:57.101929 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:57.101940 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:57.102724 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:57.103699 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.103731 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.105304 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.105309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.107800 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.107820 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.108950 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.108959 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.109461 (+ 502us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.110342 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:57.110366 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:57.113596 (+ 3230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:57.117831 (+ 4235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:57.117845 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:57.117870 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:57.118666 (+ 796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:57.119207 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:57.119213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:57.119217 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:57.119552 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:57.120329 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:57.120339 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:57.120786 (+ 447us) client_negotiation.cc:769] Sending connection context
0808 02:43:57.121145 (+ 359us) client_negotiation.cc:240] Negotiation successful
0808 02:43:57.121819 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":334,"threads_started":1}
I20260808 02:43:57.139285 29240 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:43933
I20260808 02:43:57.139326 29449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:43933 every 8 connection(s)
I20260808 02:43:57.140607 29240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:43:57.141458 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 29240
I20260808 02:43:57.141671 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:43:57.142612 26151 external_mini_cluster.cc:1428] Setting key 01ac3e0133f855be3d2a1a627dae004c
I20260808 02:43:57.144347 29453 heartbeater.cc:347] Connected to a master server at 127.25.137.254:42093
I20260808 02:43:57.144660 29453 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:57.145630 29453 heartbeater.cc:511] Master 127.25.137.254:42093 requested a full tablet report, sending...
I20260808 02:43:57.147675 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.130752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:45017 (local address 127.25.137.254:42093)
0808 02:43:57.130936 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:57.130942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:57.131636 (+ 694us) server_negotiation.cc:407] Connection header received
0808 02:43:57.132337 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:57.132343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:57.132436 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:57.132555 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:57.133994 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.135253 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.137380 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.137826 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.137949 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.140119 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:57.140146 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:57.140151 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:57.140198 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:57.143715 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.144628 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.144633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.144637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.144968 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.145510 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.145516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.145520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.146334 (+ 814us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:57.146484 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:57.147109 (+ 625us) server_negotiation.cc:299] Negotiation successful
0808 02:43:57.147351 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:43:57.148032 29426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.130843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42093 (local address 127.25.137.194:45017)
0808 02:43:57.131521 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:57.131542 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:57.131703 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:57.132628 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:57.132635 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:57.133297 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:57.133801 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.133819 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.135423 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.135428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.137208 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.137235 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.138054 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.138062 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.138248 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.138939 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:57.138966 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:57.139918 (+ 952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:57.143882 (+ 3964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:57.143888 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:57.143891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:57.144472 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:57.145156 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:57.145161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:57.145165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:57.145338 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:57.146562 (+ 1224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:57.146568 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:57.146817 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 02:43:57.146990 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 02:43:57.147359 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":367,"threads_started":1}
I20260808 02:43:57.149590 28931 ts_manager.cc:194] Registered new tserver with Master: cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933)
I20260808 02:43:57.150401 28931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:54631
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:43:57.193552 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:42093
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:43:57.527133 29459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:43:57.527436 29459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:43:57.527591 29459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:43:57.535099 29459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:43:57.535212 29459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:43:57.535337 29459 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:43:57.535408 29459 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:43:57.535503 29459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:43:57.545293 29459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.137.254:42093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:43:57.546610 29459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:43:57.547937 29459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:43:57.560544 29471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:57.560621 29468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:43:57.560822 29466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:43:57.561534 29459 server_base.cc:1034] running on GCE node
I20260808 02:43:57.562150 29459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:43:57.563107 29459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:43:57.564342 29459 hybrid_clock.cc:648] HybridClock initialized: now 1786157037564285 us; error 53 us; skew 500 ppm
I20260808 02:43:57.564652 29459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:43:57 dist-test-slave-k5rr krb5kdc[28865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157037, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:43:57.567881 29459 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:43:57.569356 29459 webserver.cc:460] Webserver started at http://127.25.137.195:40057/ using document root <none> and password file <none>
I20260808 02:43:57.570004 29459 fs_manager.cc:359] Metadata directory not provided
I20260808 02:43:57.570103 29459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:43:57.570377 29459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:43:57.573462 29459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "199527ca6fb04c479c200b0fe38b17ec"
format_stamp: "Formatted at 2026-08-08 02:43:57 on dist-test-slave-k5rr"
server_key: "26a76197abd97616c68a8707fd639834"
server_key_iv: "f2400e33b5f46303f2176a842eb85510"
server_key_version: "encryptionkey@0"
I20260808 02:43:57.574203 29459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "199527ca6fb04c479c200b0fe38b17ec"
format_stamp: "Formatted at 2026-08-08 02:43:57 on dist-test-slave-k5rr"
server_key: "26a76197abd97616c68a8707fd639834"
server_key_iv: "f2400e33b5f46303f2176a842eb85510"
server_key_version: "encryptionkey@0"
I20260808 02:43:57.579232 29459 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 02:43:57.582643 29481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:57.583912 29459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:43:57.584050 29459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "199527ca6fb04c479c200b0fe38b17ec"
format_stamp: "Formatted at 2026-08-08 02:43:57 on dist-test-slave-k5rr"
server_key: "26a76197abd97616c68a8707fd639834"
server_key_iv: "f2400e33b5f46303f2176a842eb85510"
server_key_version: "encryptionkey@0"
I20260808 02:43:57.584242 29459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:43:57.601776 29459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:43:57.605592 29459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:43:57.605860 29459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:43:57.608559 29459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:43:57.608667 29459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:43:57.608794 29459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:43:57.608875 29459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:43:57 dist-test-slave-k5rr krb5kdc[28865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157037, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:57.632387 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.609345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:58159 (local address 127.25.137.254:42093)
0808 02:43:57.609530 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:57.609537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:57.611018 (+ 1481us) server_negotiation.cc:407] Connection header received
0808 02:43:57.614015 (+ 2997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:57.614022 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:57.614119 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:57.614204 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:57.616905 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.617946 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.620254 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.620724 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.620860 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.625529 (+ 4669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:57.625535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:57.625538 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:57.625580 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:57.628966 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.629846 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.629851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.629855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.630032 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.630431 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.630436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.630440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.630829 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:57.630991 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:57.631677 (+ 686us) server_negotiation.cc:299] Negotiation successful
0808 02:43:57.631943 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:43:57.633219 29497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.609889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42093 (local address 127.25.137.195:58159)
0808 02:43:57.610840 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:57.610890 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:57.613633 (+ 2743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:57.614823 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:57.614838 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:57.615721 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:57.616685 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.616722 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.618170 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.618174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.620094 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.620107 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.620940 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.620946 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.621175 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.621892 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:57.621923 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:57.625312 (+ 3389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:57.629133 (+ 3821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:57.629142 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:57.629160 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:57.629715 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:57.630140 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:57.630143 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:57.630145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:57.630340 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:57.630967 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:57.630973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:57.631291 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 02:43:57.631553 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 02:43:57.631930 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":382,"threads_started":1}
I20260808 02:43:57.647362 29235 heartbeater.cc:503] Master 127.25.137.254:42093 was elected leader, sending a full tablet report...
I20260808 02:43:57.657959 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.644243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:58985 (local address 127.25.137.254:42093)
0808 02:43:57.644453 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:57.644458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:57.644474 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:43:57.644519 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:57.644523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:57.644578 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:57.644665 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:57.645909 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.646841 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.648529 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.649200 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.649346 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.651797 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:57.651802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:57.651806 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:57.651838 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:57.654776 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.655608 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.655614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.655618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.655779 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.656309 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.656314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.656318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.656689 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:57.656832 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:57.657392 (+ 560us) server_negotiation.cc:299] Negotiation successful
0808 02:43:57.657614 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:43:57.658087 29602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.638874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42093 (local address 127.25.137.195:58985)
0808 02:43:57.639465 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:57.639480 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:57.639580 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:57.644718 (+ 5138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:57.644722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:57.645260 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:57.645747 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.645762 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.646971 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.646976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.648391 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.648405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.649605 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.649612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.649794 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.650392 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:57.650416 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:57.651539 (+ 1123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:57.654902 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:57.654908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:57.654916 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:57.655461 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:57.655931 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:57.655936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:57.655941 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:57.656122 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:57.656841 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:57.656846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:57.657109 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 02:43:57.657281 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 02:43:57.657594 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":298,"threads_started":1}
I20260808 02:43:57.662746 29459 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:37901
I20260808 02:43:57.662777 29667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:37901 every 8 connection(s)
I20260808 02:43:57.664072 29459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:43:57.666014 29668 heartbeater.cc:347] Connected to a master server at 127.25.137.254:42093
I20260808 02:43:57.666229 29668 heartbeater.cc:464] Registering TS with master...
I20260808 02:43:57.666846 29668 heartbeater.cc:511] Master 127.25.137.254:42093 requested a full tablet report, sending...
I20260808 02:43:57.668066 28931 ts_manager.cc:194] Registered new tserver with Master: 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901)
I20260808 02:43:57.668902 28931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:58159
I20260808 02:43:57.673141 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 29459
I20260808 02:43:57.673333 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:43:57.673763 26151 external_mini_cluster.cc:1428] Setting key 0c8d4bbd81f35c3ceca0ad2dd749b21e
I20260808 02:43:57.676614 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:43:57 dist-test-slave-k5rr krb5kdc[28865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:43:57 dist-test-slave-k5rr krb5kdc[28865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:57.729228 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.715710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35868 (local address 127.25.137.254:42093)
0808 02:43:57.715976 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:57.715982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:57.716004 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:43:57.716198 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:57.716202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:57.716281 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:57.716379 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:57.717861 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.718584 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.719814 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.720236 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.720346 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.724137 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:57.724142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:57.724145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:57.724187 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:57.726617 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.727236 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.727240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.727243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.727365 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.727719 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.727724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.727727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.728013 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:57.728253 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:57.728563 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 02:43:57.728783 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260808 02:43:57.733613 28931 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:43:57.740803 28931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:43:57.773808 29687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.752733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:37901 (local address 127.0.0.1:40146)
0808 02:43:57.764916 (+ 12183us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:57.764957 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:57.765159 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:57.766808 (+ 1649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:57.766813 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:57.766850 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:57.767350 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.767364 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.769215 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.769220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.771339 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.771353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.773028 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.773036 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.773206 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.773236 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 02:43:57.773319 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 02:43:57.773417 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11899,"thread_start_us":12589,"threads_started":1}
I20260808 02:43:57.774191 29689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.753116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40146 (local address 127.25.137.195:37901)
0808 02:43:57.765992 (+ 12876us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:57.765998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:57.766024 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:43:57.766100 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:57.766105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:57.766322 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:57.766527 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:57.767596 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.769057 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.771464 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.772737 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.772946 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.773153 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:57.773681 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 02:43:57.773865 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12740,"thread_start_us":12555,"threads_started":1}
I20260808 02:43:57.774426 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.753185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49942 (local address 127.25.137.194:43933)
0808 02:43:57.765574 (+ 12389us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:57.765585 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:57.765831 (+ 246us) server_negotiation.cc:407] Connection header received
0808 02:43:57.765943 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:57.765951 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:57.766204 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:57.767217 (+ 1013us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:57.767381 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.768610 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.771415 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.772476 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.772649 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.772844 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:57.773970 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0808 02:43:57.774140 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12232,"thread_start_us":12113,"threads_started":1}
I20260808 02:43:57.773857 29686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.752921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:43933 (local address 127.0.0.1:49942)
0808 02:43:57.765568 (+ 12647us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:57.765590 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:57.765704 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:57.766474 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:57.766481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:57.766513 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:57.767000 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.767015 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.768731 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.768736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.771218 (+ 2482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.771234 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.772669 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.772675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.772857 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.772909 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 02:43:57.773118 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 02:43:57.773349 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12535,"mutex_wait_us":2,"thread_start_us":11574,"threads_started":1}
I20260808 02:43:57.774737 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.766022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.25.137.193:41301)
0808 02:43:57.766617 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:57.766626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:57.767444 (+ 818us) server_negotiation.cc:407] Connection header received
0808 02:43:57.767601 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:57.767608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:57.767821 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:57.768001 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:57.768583 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.769914 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.771907 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.773321 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.773492 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.773721 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:57.774276 (+ 555us) server_negotiation.cc:299] Negotiation successful
0808 02:43:57.774453 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":296,"threads_started":1}
I20260808 02:43:57.775355 29692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.766624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:41301 (local address 127.0.0.1:44926)
0808 02:43:57.767310 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:57.767331 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:57.767456 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:57.768015 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:57.768019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:57.768045 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:57.768414 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.768425 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.770075 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.770081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.771733 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.771749 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.773891 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.773897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.774066 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.774097 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:43:57.774907 (+ 810us) client_negotiation.cc:240] Negotiation successful
0808 02:43:57.775040 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":361,"threads_started":1}
I20260808 02:43:57.778093 29563 tablet_service.cc:1467] Processing CreateTablet for tablet 8d053007921d48528f74e9215f0b4bb0 (DEFAULT_TABLE table=test-table [id=d3202a2d8882495ebdce3f6ec41b0fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:57.778252 29346 tablet_service.cc:1467] Processing CreateTablet for tablet 8d053007921d48528f74e9215f0b4bb0 (DEFAULT_TABLE table=test-table [id=d3202a2d8882495ebdce3f6ec41b0fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:57.778729 29133 tablet_service.cc:1467] Processing CreateTablet for tablet 8d053007921d48528f74e9215f0b4bb0 (DEFAULT_TABLE table=test-table [id=d3202a2d8882495ebdce3f6ec41b0fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:57.779327 29563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d053007921d48528f74e9215f0b4bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:57.779580 29346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d053007921d48528f74e9215f0b4bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:57.780019 29133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d053007921d48528f74e9215f0b4bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:57.791083 29694 tablet_bootstrap.cc:492] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: Bootstrap starting.
I20260808 02:43:57.795096 29694 tablet_bootstrap.cc:654] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:57.795924 29698 tablet_bootstrap.cc:492] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: Bootstrap starting.
I20260808 02:43:57.796382 29694 log.cc:824] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:57.796762 29695 tablet_bootstrap.cc:492] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: Bootstrap starting.
I20260808 02:43:57.799697 29698 tablet_bootstrap.cc:654] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:57.799947 29695 tablet_bootstrap.cc:654] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:57.800216 29694 tablet_bootstrap.cc:492] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: No bootstrap required, opened a new log
I20260808 02:43:57.800506 29694 ts_tablet_manager.cc:1397] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20260808 02:43:57.800797 29698 log.cc:824] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:57.801169 29695 log.cc:824] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:43:57.803649 29698 tablet_bootstrap.cc:492] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: No bootstrap required, opened a new log
I20260808 02:43:57.803925 29698 ts_tablet_manager.cc:1397] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 02:43:57.804015 29695 tablet_bootstrap.cc:492] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: No bootstrap required, opened a new log
I20260808 02:43:57.804267 29695 ts_tablet_manager.cc:1397] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 02:43:57.811195 29695 raft_consensus.cc:348] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } }
I20260808 02:43:57.811198 29694 raft_consensus.cc:348] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } }
I20260808 02:43:57.811582 29695 raft_consensus.cc:374] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:57.811695 29694 raft_consensus.cc:374] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:57.811705 29695 raft_consensus.cc:729] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd859eb96e974fba869481bce2d02bb1, State: Initialized, Role: FOLLOWER
I20260808 02:43:57.811802 29694 raft_consensus.cc:729] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 199527ca6fb04c479c200b0fe38b17ec, State: Initialized, Role: FOLLOWER
I20260808 02:43:57.812346 29695 consensus_queue.cc:260] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } }
I20260808 02:43:57.812397 29694 consensus_queue.cc:260] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } }
I20260808 02:43:57.813766 29453 heartbeater.cc:503] Master 127.25.137.254:42093 was elected leader, sending a full tablet report...
I20260808 02:43:57.814401 29695 ts_tablet_manager.cc:1428] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: Time spent starting tablet: real 0.010s user 0.008s sys 0.001s
I20260808 02:43:57.814679 29698 raft_consensus.cc:348] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } }
I20260808 02:43:57.815193 29698 raft_consensus.cc:374] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:57.815326 29698 raft_consensus.cc:729] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3080a7113294155897915ae876400c7, State: Initialized, Role: FOLLOWER
I20260808 02:43:57.815668 29668 heartbeater.cc:503] Master 127.25.137.254:42093 was elected leader, sending a full tablet report...
I20260808 02:43:57.816076 29694 ts_tablet_manager.cc:1428] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260808 02:43:57.815997 29698 consensus_queue.cc:260] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } }
I20260808 02:43:57.818682 29698 ts_tablet_manager.cc:1428] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
W20260808 02:43:57.890987 29236 tablet.cc:2369] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:43:57.897930 29713 raft_consensus.cc:482] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:43:57.898200 29713 raft_consensus.cc:504] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } }
W20260808 02:43:57.898615 29454 tablet.cc:2369] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:43:57.899894 29713 leader_election.cc:288] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933), c3080a7113294155897915ae876400c7 (127.25.137.193:41301)
I20260808 02:43:57.906180 29497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.900388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:41301 (local address 127.25.137.195:44881)
0808 02:43:57.900568 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:57.900588 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:57.900709 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:57.901088 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:57.901093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:57.901135 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:57.901421 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.901431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.902611 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.902615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.904346 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.904369 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.905567 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.905571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.905720 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.905779 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 02:43:57.905865 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:43:57.905964 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 02:43:57.906268 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.900314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:44881 (local address 127.25.137.193:41301)
0808 02:43:57.900536 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:57.900542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:57.900662 (+ 120us) server_negotiation.cc:407] Connection header received
0808 02:43:57.900849 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:57.900854 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:57.900927 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:57.901053 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:57.901570 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.902486 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.904528 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.905413 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.905570 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.905644 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:57.905903 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 02:43:57.906033 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:43:57.907474 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.900394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:43571 (local address 127.25.137.194:43933)
0808 02:43:57.900569 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:57.900574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:57.901652 (+ 1078us) server_negotiation.cc:407] Connection header received
0808 02:43:57.901759 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:57.901764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:57.901841 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:57.901933 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:57.902379 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.903146 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.905368 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.906421 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.906573 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.906639 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:57.907045 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 02:43:57.907192 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:43:57.907502 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.900406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:43933 (local address 127.25.137.195:43571)
0808 02:43:57.901552 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:57.901566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:57.901665 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:57.901949 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:57.901952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:57.901970 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:57.902239 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.902247 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.903293 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.903298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.905172 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.905187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.906649 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.906655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.906813 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.906850 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:43:57.906960 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 02:43:57.907105 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1040,"mutex_wait_us":373,"thread_start_us":413,"threads_started":1}
I20260808 02:43:57.907392 29172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d053007921d48528f74e9215f0b4bb0" candidate_uuid: "199527ca6fb04c479c200b0fe38b17ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3080a7113294155897915ae876400c7" is_pre_election: true
I20260808 02:43:57.908084 29172 raft_consensus.cc:2454] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 199527ca6fb04c479c200b0fe38b17ec in term 0.
I20260808 02:43:57.908465 29372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d053007921d48528f74e9215f0b4bb0" candidate_uuid: "199527ca6fb04c479c200b0fe38b17ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd859eb96e974fba869481bce2d02bb1" is_pre_election: true
I20260808 02:43:57.908938 29488 leader_election.cc:302] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 199527ca6fb04c479c200b0fe38b17ec, c3080a7113294155897915ae876400c7; no voters:
I20260808 02:43:57.909044 29372 raft_consensus.cc:2454] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 199527ca6fb04c479c200b0fe38b17ec in term 0.
I20260808 02:43:57.909366 29713 raft_consensus.cc:2781] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:43:57.909521 29713 raft_consensus.cc:482] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:43:57.909602 29713 raft_consensus.cc:3037] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:57.911860 29713 raft_consensus.cc:504] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } }
I20260808 02:43:57.912465 29713 leader_election.cc:288] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [CANDIDATE]: Term 1 election: Requested vote from peers cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933), c3080a7113294155897915ae876400c7 (127.25.137.193:41301)
I20260808 02:43:57.912990 29372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d053007921d48528f74e9215f0b4bb0" candidate_uuid: "199527ca6fb04c479c200b0fe38b17ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd859eb96e974fba869481bce2d02bb1"
I20260808 02:43:57.912958 29172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d053007921d48528f74e9215f0b4bb0" candidate_uuid: "199527ca6fb04c479c200b0fe38b17ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3080a7113294155897915ae876400c7"
I20260808 02:43:57.913205 29372 raft_consensus.cc:3037] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:57.913205 29172 raft_consensus.cc:3037] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Advancing to term 1
W20260808 02:43:57.915558 29670 tablet.cc:2369] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:43:57.916363 29172 raft_consensus.cc:2454] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 199527ca6fb04c479c200b0fe38b17ec in term 1.
I20260808 02:43:57.916354 29372 raft_consensus.cc:2454] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 199527ca6fb04c479c200b0fe38b17ec in term 1.
I20260808 02:43:57.916775 29489 leader_election.cc:302] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 199527ca6fb04c479c200b0fe38b17ec, cd859eb96e974fba869481bce2d02bb1; no voters:
I20260808 02:43:57.917131 29713 raft_consensus.cc:2781] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:43:57.917574 29713 raft_consensus.cc:686] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 1 LEADER]: Becoming Leader. State: Replica: 199527ca6fb04c479c200b0fe38b17ec, State: Running, Role: LEADER
I20260808 02:43:57.918023 29713 consensus_queue.cc:237] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } }
I20260808 02:43:57.922927 28932 catalog_manager.cc:5495] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195). New cstate: current_term: 1 leader_uuid: "199527ca6fb04c479c200b0fe38b17ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:43:57 dist-test-slave-k5rr krb5kdc[28865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157035, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:43:57.984962 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.971661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35874 (local address 127.25.137.254:42093)
0808 02:43:57.971917 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:57.971923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:57.972418 (+ 495us) server_negotiation.cc:407] Connection header received
0808 02:43:57.972557 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:57.972561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:57.972621 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:57.972703 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:43:57.973926 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.974672 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.975930 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.976354 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.976464 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.979753 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:43:57.979757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:43:57.979761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:43:57.979789 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:43:57.982233 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.982883 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.982888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.982890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.983011 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:43:57.983372 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:43:57.983378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:43:57.983381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:43:57.983653 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:43:57.983791 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:57.984257 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 02:43:57.984590 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":76}
I20260808 02:43:57.984962 29726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:57.971660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42093 (local address 127.0.0.1:35874)
0808 02:43:57.972312 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:57.972329 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:57.972462 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:57.972738 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:57.972741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:57.973517 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:43:57.973783 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.973793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.974788 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.974792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.975788 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:57.975801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:57.976538 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:57.976544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:57.976685 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:57.977347 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:43:57.977365 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:43:57.979575 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:43:57.982356 (+ 2781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:57.982370 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:57.982379 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:57.982775 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:57.983119 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:43:57.983124 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:43:57.983127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:43:57.983264 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:43:57.983782 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:43:57.983790 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:43:57.983994 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 02:43:57.984152 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 02:43:57.984442 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":365,"threads_started":1}
I20260808 02:43:57.990980 28932 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:35874:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:43:57.992102 28932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:43:57.999060 29133 tablet_service.cc:1467] Processing CreateTablet for tablet f56e73c8ef2f41669c8240223ab423b5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52811937f2724df49163b543eb0f1b36]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:43:57.999358 29563 tablet_service.cc:1467] Processing CreateTablet for tablet f56e73c8ef2f41669c8240223ab423b5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52811937f2724df49163b543eb0f1b36]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:43:57.999382 29346 tablet_service.cc:1467] Processing CreateTablet for tablet f56e73c8ef2f41669c8240223ab423b5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52811937f2724df49163b543eb0f1b36]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:43:58.000141 29563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f56e73c8ef2f41669c8240223ab423b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:58.000156 29346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f56e73c8ef2f41669c8240223ab423b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:58.000159 29133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f56e73c8ef2f41669c8240223ab423b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:58.006227 29694 tablet_bootstrap.cc:492] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec: Bootstrap starting.
I20260808 02:43:58.007335 29695 tablet_bootstrap.cc:492] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: Bootstrap starting.
I20260808 02:43:58.007815 29694 tablet_bootstrap.cc:654] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:58.008285 29698 tablet_bootstrap.cc:492] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7: Bootstrap starting.
I20260808 02:43:58.009060 29695 tablet_bootstrap.cc:654] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:58.009943 29698 tablet_bootstrap.cc:654] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:58.010330 29694 tablet_bootstrap.cc:492] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec: No bootstrap required, opened a new log
I20260808 02:43:58.010475 29694 ts_tablet_manager.cc:1397] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260808 02:43:58.011268 29694 raft_consensus.cc:348] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.011620 29694 raft_consensus.cc:374] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:58.011731 29694 raft_consensus.cc:729] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 199527ca6fb04c479c200b0fe38b17ec, State: Initialized, Role: FOLLOWER
I20260808 02:43:58.012015 29695 tablet_bootstrap.cc:492] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: No bootstrap required, opened a new log
I20260808 02:43:58.011976 29694 consensus_queue.cc:260] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.012154 29695 ts_tablet_manager.cc:1397] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 02:43:58.012657 29694 ts_tablet_manager.cc:1428] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260808 02:43:58.012487 29713 tablet_replica.cc:439] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } }
I20260808 02:43:58.012894 29713 tablet_replica.cc:442] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:58.012974 29695 raft_consensus.cc:348] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.013510 29695 raft_consensus.cc:374] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:58.013666 29695 raft_consensus.cc:729] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd859eb96e974fba869481bce2d02bb1, State: Initialized, Role: FOLLOWER
I20260808 02:43:58.014004 29695 consensus_queue.cc:260] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.014534 29708 tablet_replica.cc:439] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } }
I20260808 02:43:58.014912 29708 tablet_replica.cc:442] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:58.015084 29698 tablet_bootstrap.cc:492] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7: No bootstrap required, opened a new log
I20260808 02:43:58.015287 29698 ts_tablet_manager.cc:1397] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260808 02:43:58.015501 29695 ts_tablet_manager.cc:1428] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:43:58.016108 29698 raft_consensus.cc:348] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.016366 29698 raft_consensus.cc:374] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:58.016484 29698 raft_consensus.cc:729] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3080a7113294155897915ae876400c7, State: Initialized, Role: FOLLOWER
I20260808 02:43:58.016709 29698 consensus_queue.cc:260] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.017335 29698 ts_tablet_manager.cc:1428] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:43:58.017196 29714 tablet_replica.cc:439] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } }
I20260808 02:43:58.017611 29714 tablet_replica.cc:442] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:58.227085 29708 raft_consensus.cc:482] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:43:58.227363 29708 raft_consensus.cc:504] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.229066 29708 leader_election.cc:288] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3080a7113294155897915ae876400c7 (127.25.137.193:41301), 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901)
I20260808 02:43:58.236332 29714 raft_consensus.cc:482] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:43:58.236589 29714 raft_consensus.cc:504] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.238291 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.229591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:37009 (local address 127.25.137.193:41301)
0808 02:43:58.229791 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:58.229797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:58.230458 (+ 661us) server_negotiation.cc:407] Connection header received
0808 02:43:58.230793 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:58.230799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:58.230887 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:58.230950 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:58.231763 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.232969 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.235716 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.236999 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.237183 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.237256 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:58.237831 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 02:43:58.237967 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:43:58.238447 29689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.230642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:41055 (local address 127.25.137.195:37901)
0808 02:43:58.230830 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:58.230836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:58.231346 (+ 510us) server_negotiation.cc:407] Connection header received
0808 02:43:58.231566 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:58.231571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:58.231649 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:58.231741 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:58.232239 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.232933 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.236170 (+ 3237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.237338 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.237463 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.237532 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:58.238060 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 02:43:58.238195 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:43:58.238530 29714 leader_election.cc:288] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933), 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901)
I20260808 02:43:58.239146 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.229415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:41301 (local address 127.25.137.194:37009)
0808 02:43:58.230350 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:58.230369 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:58.230590 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:58.231145 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:58.231148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:58.231182 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:58.231582 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.231597 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.233132 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.233136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.235523 (+ 2387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.235541 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.237448 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.237454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.237624 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.237661 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:43:58.238537 (+ 876us) client_negotiation.cc:240] Negotiation successful
0808 02:43:58.238680 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":115,"thread_start_us":558,"threads_started":1}
I20260808 02:43:58.239166 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.230470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:37901 (local address 127.25.137.194:41055)
0808 02:43:58.231185 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:58.231213 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:58.231393 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:58.231774 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:58.231778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:58.231804 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:58.232091 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.232099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.233081 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.233087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.235937 (+ 2850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.235952 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.237716 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.237722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.237877 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.237910 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:43:58.238598 (+ 688us) client_negotiation.cc:240] Negotiation successful
0808 02:43:58.238701 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":64,"thread_start_us":370,"threads_started":1}
I20260808 02:43:58.239926 29172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56e73c8ef2f41669c8240223ab423b5" candidate_uuid: "cd859eb96e974fba869481bce2d02bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3080a7113294155897915ae876400c7" is_pre_election: true
I20260808 02:43:58.240192 29172 raft_consensus.cc:2454] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd859eb96e974fba869481bce2d02bb1 in term 0.
I20260808 02:43:58.240370 29605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56e73c8ef2f41669c8240223ab423b5" candidate_uuid: "cd859eb96e974fba869481bce2d02bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "199527ca6fb04c479c200b0fe38b17ec" is_pre_election: true
I20260808 02:43:58.240736 29605 raft_consensus.cc:2454] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd859eb96e974fba869481bce2d02bb1 in term 0.
I20260808 02:43:58.241333 29269 leader_election.cc:302] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3080a7113294155897915ae876400c7, cd859eb96e974fba869481bce2d02bb1; no voters:
I20260808 02:43:58.242092 29708 raft_consensus.cc:2781] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:43:58.242208 29708 raft_consensus.cc:482] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:43:58.242309 29708 raft_consensus.cc:3037] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:58.245528 29708 raft_consensus.cc:504] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.246801 29172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56e73c8ef2f41669c8240223ab423b5" candidate_uuid: "cd859eb96e974fba869481bce2d02bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3080a7113294155897915ae876400c7"
I20260808 02:43:58.247038 29172 raft_consensus.cc:3037] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:58.248755 29689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.240683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:43321 (local address 127.25.137.195:37901)
0808 02:43:58.240854 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:58.240859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:58.241371 (+ 512us) server_negotiation.cc:407] Connection header received
0808 02:43:58.241571 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:58.241576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:58.241652 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:58.241740 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:58.242542 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.243572 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.246431 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.247546 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.247666 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.247733 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:58.248366 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 02:43:58.248498 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:43:58.249454 29738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.239391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:37901 (local address 127.25.137.193:43321)
0808 02:43:58.241266 (+ 1875us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:58.241284 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:58.241415 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:58.241908 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:58.241912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:58.241940 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:58.242351 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.242366 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.243758 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.243763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.246256 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.246272 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.247960 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.247968 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.248177 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.248213 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:43:58.248932 (+ 719us) client_negotiation.cc:240] Negotiation successful
0808 02:43:58.249102 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1764,"threads_started":1}
I20260808 02:43:58.250116 29172 raft_consensus.cc:2454] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd859eb96e974fba869481bce2d02bb1 in term 1.
I20260808 02:43:58.250684 29605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56e73c8ef2f41669c8240223ab423b5" candidate_uuid: "c3080a7113294155897915ae876400c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "199527ca6fb04c479c200b0fe38b17ec" is_pre_election: true
I20260808 02:43:58.250836 29269 leader_election.cc:302] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3080a7113294155897915ae876400c7, cd859eb96e974fba869481bce2d02bb1; no voters:
I20260808 02:43:58.250941 29605 raft_consensus.cc:2454] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3080a7113294155897915ae876400c7 in term 0.
I20260808 02:43:58.251681 29060 leader_election.cc:302] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 199527ca6fb04c479c200b0fe38b17ec, c3080a7113294155897915ae876400c7; no voters:
I20260808 02:43:58.252131 29714 raft_consensus.cc:2741] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260808 02:43:58.253180 29742 raft_consensus.cc:2781] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:43:58.253386 29708 leader_election.cc:288] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [CANDIDATE]: Term 1 election: Requested vote from peers c3080a7113294155897915ae876400c7 (127.25.137.193:41301), 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901)
I20260808 02:43:58.253925 29742 raft_consensus.cc:686] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [term 1 LEADER]: Becoming Leader. State: Replica: cd859eb96e974fba869481bce2d02bb1, State: Running, Role: LEADER
I20260808 02:43:58.254763 29605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56e73c8ef2f41669c8240223ab423b5" candidate_uuid: "cd859eb96e974fba869481bce2d02bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "199527ca6fb04c479c200b0fe38b17ec"
I20260808 02:43:58.254590 29742 consensus_queue.cc:237] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.254993 29605 raft_consensus.cc:3037] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:58.258318 29605 raft_consensus.cc:2454] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd859eb96e974fba869481bce2d02bb1 in term 1.
I20260808 02:43:58.258837 29737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.240629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:43933 (local address 127.25.137.193:35033)
0808 02:43:58.249635 (+ 9006us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:58.249655 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:58.249799 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:58.250348 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:58.250353 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:58.250381 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:43:58.250708 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.250719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.254628 (+ 3909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.254634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.256541 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.256557 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.258135 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.258139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.258260 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.258282 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:43:58.258373 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 02:43:58.258504 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8869,"thread_start_us":368,"threads_started":1}
I20260808 02:43:58.258855 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.240396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:35033 (local address 127.25.137.194:43933)
0808 02:43:58.240561 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:58.240566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:58.249759 (+ 9193us) server_negotiation.cc:407] Connection header received
0808 02:43:58.249917 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:58.249922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:58.249998 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:58.250090 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:43:58.250924 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.254084 (+ 3160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.256720 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.257806 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.257923 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.257975 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:58.258427 (+ 452us) server_negotiation.cc:299] Negotiation successful
0808 02:43:58.258559 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:43:58.259696 29372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56e73c8ef2f41669c8240223ab423b5" candidate_uuid: "c3080a7113294155897915ae876400c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd859eb96e974fba869481bce2d02bb1" is_pre_election: true
I20260808 02:43:58.260922 29708 tablet_replica.cc:439] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: TxnStatusTablet state changed. Reason: New leader cd859eb96e974fba869481bce2d02bb1. Latest consensus state: current_term: 1 leader_uuid: "cd859eb96e974fba869481bce2d02bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } }
I20260808 02:43:58.261227 29708 tablet_replica.cc:442] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:43:58.261767 29747 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:43:58.262029 29747 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:43:58.262319 28930 catalog_manager.cc:5495] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd859eb96e974fba869481bce2d02bb1 (127.25.137.194). New cstate: current_term: 1 leader_uuid: "cd859eb96e974fba869481bce2d02bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } health_report { overall_health: UNKNOWN } } }
I20260808 02:43:58.314658 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.307380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49956 (local address 127.25.137.194:43933)
0808 02:43:58.307606 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:58.307612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:58.307634 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:43:58.307935 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:58.307941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:58.308046 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:58.308217 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:43:58.308622 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.309937 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.311936 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.312597 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.312781 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.313259 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:43:58.313975 (+ 716us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:43:58.314127 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:58.314242 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 02:43:58.314331 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 02:43:58.314674 29726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.307298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:43933 (local address 127.0.0.1:49956)
0808 02:43:58.307545 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:58.307567 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:58.307706 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:58.308251 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:58.308256 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:58.308277 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:43:58.308473 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.308485 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.310108 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.310113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.311750 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.311769 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.312760 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.312767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.312961 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.313060 (+ 99us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:43:58.314096 (+ 1036us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:43:58.314127 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:43:58.314219 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 02:43:58.314382 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 02:43:58.323467 29605 raft_consensus.cc:1261] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Refusing update from remote peer cd859eb96e974fba869481bce2d02bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:43:58.323525 29172 raft_consensus.cc:1261] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Refusing update from remote peer cd859eb96e974fba869481bce2d02bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:43:58.324419 29742 consensus_queue.cc:1035] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:43:58.324805 29708 consensus_queue.cc:1035] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:43:58.347198 29714 tablet_replica.cc:439] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7: TxnStatusTablet state changed. Reason: New leader cd859eb96e974fba869481bce2d02bb1. Latest consensus state: current_term: 1 leader_uuid: "cd859eb96e974fba869481bce2d02bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } }
I20260808 02:43:58.347545 29714 tablet_replica.cc:442] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:58.350102 29750 mvcc.cc:205] Tried to move back new op lower bound from 7316099228962721792 to 7316099228695867392. Current Snapshot: MvccSnapshot[applied={T|T < 7316099228962721792}]
I20260808 02:43:58.351691 29716 consensus_queue.cc:1035] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:43:58.357651 29713 tablet_replica.cc:439] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec: TxnStatusTablet state changed. Reason: New leader cd859eb96e974fba869481bce2d02bb1. Latest consensus state: current_term: 1 leader_uuid: "cd859eb96e974fba869481bce2d02bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } }
I20260808 02:43:58.357998 29713 tablet_replica.cc:442] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:58.362167 29743 tablet_replica.cc:439] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd859eb96e974fba869481bce2d02bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } }
I20260808 02:43:58.362692 29743 tablet_replica.cc:442] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:43:58.364162 29742 tablet_replica.cc:439] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd859eb96e974fba869481bce2d02bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } }
I20260808 02:43:58.364434 29742 tablet_replica.cc:442] T f56e73c8ef2f41669c8240223ab423b5 P cd859eb96e974fba869481bce2d02bb1: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:43:58.369361 29716 consensus_queue.cc:1035] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:43:58.369573 29714 tablet_replica.cc:439] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd859eb96e974fba869481bce2d02bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } }
I20260808 02:43:58.369817 29714 tablet_replica.cc:442] T f56e73c8ef2f41669c8240223ab423b5 P c3080a7113294155897915ae876400c7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:58.376933 29716 tablet_replica.cc:439] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd859eb96e974fba869481bce2d02bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } } }
I20260808 02:43:58.377326 29716 tablet_replica.cc:442] T f56e73c8ef2f41669c8240223ab423b5 P 199527ca6fb04c479c200b0fe38b17ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:43:58.383749 29689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.371844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.25.137.195:37901)
0808 02:43:58.372019 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:58.372026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:58.373401 (+ 1375us) server_negotiation.cc:407] Connection header received
0808 02:43:58.373718 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:58.373724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:58.373824 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:58.373941 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:43:58.374589 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.375775 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.381053 (+ 5278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.381691 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.381821 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.382461 (+ 640us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:43:58.383163 (+ 702us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:43:58.383298 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:58.383392 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 02:43:58.383473 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:43:58.400887 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.394622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:36735 (local address 127.25.137.194:43933)
0808 02:43:58.394858 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:58.394864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:58.395337 (+ 473us) server_negotiation.cc:407] Connection header received
0808 02:43:58.395531 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:58.395535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:58.395608 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:58.395678 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:43:58.396249 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.397036 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.399070 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.399553 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.399690 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.400210 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:43:58.400358 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:43:58.400437 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:58.400573 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 02:43:58.400642 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:43:58.401214 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.394600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:43933 (local address 127.25.137.195:36735)
0808 02:43:58.395233 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:58.395254 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:58.395412 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:58.395717 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:58.395722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:58.395735 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:43:58.396049 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.396061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.397166 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.397172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.398908 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.398924 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.399729 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.399736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.399907 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.399981 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:43:58.400457 (+ 476us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:43:58.400468 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 02:43:58.400544 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:43:58.400690 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":303,"threads_started":1}
I20260808 02:43:58.441042 29689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.433633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:55859 (local address 127.25.137.195:37901)
0808 02:43:58.433824 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:58.433830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:58.434626 (+ 796us) server_negotiation.cc:407] Connection header received
0808 02:43:58.434924 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:58.434930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:58.435011 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:58.435099 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:43:58.435973 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.437092 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.439320 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.439762 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.439866 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.440394 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:43:58.440497 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:43:58.440562 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:58.440694 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 02:43:58.440756 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:43:58.441257 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.433733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:37901 (local address 127.25.137.194:55859)
0808 02:43:58.434517 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:58.434538 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:58.434754 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:58.435270 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:58.435275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:58.435289 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:43:58.435717 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.435730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.437272 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.437277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.439146 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.439162 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.439948 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.439954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.440130 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.440212 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:43:58.440590 (+ 378us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:43:58.440602 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 02:43:58.440683 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 02:43:58.440826 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":1162,"threads_started":1}
I20260808 02:43:58.478583 28930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:43:58.478914 28930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:43:58.483229 28930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70: Sending DeleteTablet for 3 replicas of tablet 8d053007921d48528f74e9215f0b4bb0
I20260808 02:43:58.484472 29563 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d053007921d48528f74e9215f0b4bb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:43:58 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:40146
I20260808 02:43:58.484480 29346 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d053007921d48528f74e9215f0b4bb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:43:58 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:49942
I20260808 02:43:58.485950 29133 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d053007921d48528f74e9215f0b4bb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:43:58 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:44926
I20260808 02:43:58.487659 28930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:43:58.489441 28930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:43:58.499127 29345 tablet_service.cc:1467] Processing CreateTablet for tablet 97159a8931c24782b6c1e66c11c8846c (DEFAULT_TABLE table=test-table [id=d757fea281f742d481bed78f5891fe2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:58.499632 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 97159a8931c24782b6c1e66c11c8846c (DEFAULT_TABLE table=test-table [id=d757fea281f742d481bed78f5891fe2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:58.499799 29345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97159a8931c24782b6c1e66c11c8846c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:58.500223 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97159a8931c24782b6c1e66c11c8846c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:58.505594 29132 tablet_service.cc:1467] Processing CreateTablet for tablet 97159a8931c24782b6c1e66c11c8846c (DEFAULT_TABLE table=test-table [id=d757fea281f742d481bed78f5891fe2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:43:58.506177 29132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97159a8931c24782b6c1e66c11c8846c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:43:58.507154 29779 tablet_replica.cc:330] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: stopping tablet replica
I20260808 02:43:58.507637 29779 raft_consensus.cc:2229] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:43:58.508347 29779 raft_consensus.cc:2258] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:43:58.509524 29695 tablet_bootstrap.cc:492] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1: Bootstrap starting.
I20260808 02:43:58.510529 29779 ts_tablet_manager.cc:1905] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:43:58.511313 29695 tablet_bootstrap.cc:654] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:58.513661 29778 tablet_replica.cc:330] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: stopping tablet replica
I20260808 02:43:58.513985 29782 tablet_bootstrap.cc:492] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec: Bootstrap starting.
I20260808 02:43:58.514123 29778 raft_consensus.cc:2229] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:43:58.514492 29778 raft_consensus.cc:2258] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:43:58.515040 29695 tablet_bootstrap.cc:492] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1: No bootstrap required, opened a new log
I20260808 02:43:58.515209 29695 ts_tablet_manager.cc:1397] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260808 02:43:58.515605 29698 tablet_bootstrap.cc:492] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7: Bootstrap starting.
I20260808 02:43:58.515995 29695 raft_consensus.cc:348] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.516233 29695 raft_consensus.cc:374] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:58.516321 29695 raft_consensus.cc:729] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd859eb96e974fba869481bce2d02bb1, State: Initialized, Role: FOLLOWER
I20260808 02:43:58.516412 29778 ts_tablet_manager.cc:1905] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:43:58.517366 29698 tablet_bootstrap.cc:654] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:58.517696 29695 consensus_queue.cc:260] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.518914 29695 ts_tablet_manager.cc:1428] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260808 02:43:58.526135 29780 tablet_replica.cc:330] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: stopping tablet replica
I20260808 02:43:58.526614 29780 raft_consensus.cc:2229] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:43:58.527056 29780 raft_consensus.cc:2258] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:43:58.527956 29698 tablet_bootstrap.cc:492] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7: No bootstrap required, opened a new log
I20260808 02:43:58.528177 29698 ts_tablet_manager.cc:1397] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260808 02:43:58.529053 29780 ts_tablet_manager.cc:1905] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:43:58.528990 29698 raft_consensus.cc:348] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.529390 29698 raft_consensus.cc:374] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:58.529520 29698 raft_consensus.cc:729] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3080a7113294155897915ae876400c7, State: Initialized, Role: FOLLOWER
I20260808 02:43:58.529968 29778 ts_tablet_manager.cc:1918] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:43:58.529865 29698 consensus_queue.cc:260] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.530328 29778 log.cc:1196] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/8d053007921d48528f74e9215f0b4bb0
I20260808 02:43:58.530921 29778 ts_tablet_manager.cc:1939] T 8d053007921d48528f74e9215f0b4bb0 P cd859eb96e974fba869481bce2d02bb1: Deleting consensus metadata
I20260808 02:43:58.532773 29698 ts_tablet_manager.cc:1428] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260808 02:43:58.532763 28911 catalog_manager.cc:4841] TS cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933): tablet 8d053007921d48528f74e9215f0b4bb0 (table test-table [id=d3202a2d8882495ebdce3f6ec41b0fca]) successfully deleted
I20260808 02:43:58.535745 29780 ts_tablet_manager.cc:1918] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:43:58.535899 29780 log.cc:1196] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/8d053007921d48528f74e9215f0b4bb0
I20260808 02:43:58.536319 29780 ts_tablet_manager.cc:1939] T 8d053007921d48528f74e9215f0b4bb0 P c3080a7113294155897915ae876400c7: Deleting consensus metadata
I20260808 02:43:58.537127 29779 ts_tablet_manager.cc:1918] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:43:58.537545 29779 log.cc:1196] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/8d053007921d48528f74e9215f0b4bb0
I20260808 02:43:58.538157 29779 ts_tablet_manager.cc:1939] T 8d053007921d48528f74e9215f0b4bb0 P 199527ca6fb04c479c200b0fe38b17ec: Deleting consensus metadata
I20260808 02:43:58.540004 28910 catalog_manager.cc:4841] TS 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901): tablet 8d053007921d48528f74e9215f0b4bb0 (table test-table [id=d3202a2d8882495ebdce3f6ec41b0fca]) successfully deleted
I20260808 02:43:58.541620 29782 tablet_bootstrap.cc:654] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec: Neither blocks nor log segments found. Creating new log.
I20260808 02:43:58.541909 28910 catalog_manager.cc:4841] TS c3080a7113294155897915ae876400c7 (127.25.137.193:41301): tablet 8d053007921d48528f74e9215f0b4bb0 (table test-table [id=d3202a2d8882495ebdce3f6ec41b0fca]) successfully deleted
I20260808 02:43:58.545469 29782 tablet_bootstrap.cc:492] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec: No bootstrap required, opened a new log
I20260808 02:43:58.545689 29782 ts_tablet_manager.cc:1397] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec: Time spent bootstrapping tablet: real 0.032s user 0.004s sys 0.000s
I20260808 02:43:58.546594 29782 raft_consensus.cc:348] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.546886 29782 raft_consensus.cc:374] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:43:58.546998 29782 raft_consensus.cc:729] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 199527ca6fb04c479c200b0fe38b17ec, State: Initialized, Role: FOLLOWER
I20260808 02:43:58.547243 29782 consensus_queue.cc:260] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.547884 29782 ts_tablet_manager.cc:1428] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:43:58.563899 29742 raft_consensus.cc:482] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:43:58.564142 29742 raft_consensus.cc:504] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.564966 29742 leader_election.cc:288] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3080a7113294155897915ae876400c7 (127.25.137.193:41301), 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901)
I20260808 02:43:58.565675 29172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97159a8931c24782b6c1e66c11c8846c" candidate_uuid: "cd859eb96e974fba869481bce2d02bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3080a7113294155897915ae876400c7" is_pre_election: true
I20260808 02:43:58.565702 29605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97159a8931c24782b6c1e66c11c8846c" candidate_uuid: "cd859eb96e974fba869481bce2d02bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "199527ca6fb04c479c200b0fe38b17ec" is_pre_election: true
I20260808 02:43:58.565963 29172 raft_consensus.cc:2454] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd859eb96e974fba869481bce2d02bb1 in term 0.
I20260808 02:43:58.565984 29605 raft_consensus.cc:2454] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd859eb96e974fba869481bce2d02bb1 in term 0.
I20260808 02:43:58.566529 29269 leader_election.cc:302] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 199527ca6fb04c479c200b0fe38b17ec, cd859eb96e974fba869481bce2d02bb1; no voters:
I20260808 02:43:58.566957 29742 raft_consensus.cc:2781] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:43:58.567096 29742 raft_consensus.cc:482] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:43:58.567193 29742 raft_consensus.cc:3037] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:58.570441 29742 raft_consensus.cc:504] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.571128 29742 leader_election.cc:288] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [CANDIDATE]: Term 1 election: Requested vote from peers c3080a7113294155897915ae876400c7 (127.25.137.193:41301), 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901)
I20260808 02:43:58.571728 29172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97159a8931c24782b6c1e66c11c8846c" candidate_uuid: "cd859eb96e974fba869481bce2d02bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3080a7113294155897915ae876400c7"
I20260808 02:43:58.571771 29605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97159a8931c24782b6c1e66c11c8846c" candidate_uuid: "cd859eb96e974fba869481bce2d02bb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "199527ca6fb04c479c200b0fe38b17ec"
I20260808 02:43:58.571976 29172 raft_consensus.cc:3037] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:58.572016 29605 raft_consensus.cc:3037] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:43:58.575059 29172 raft_consensus.cc:2454] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd859eb96e974fba869481bce2d02bb1 in term 1.
I20260808 02:43:58.575330 29605 raft_consensus.cc:2454] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd859eb96e974fba869481bce2d02bb1 in term 1.
I20260808 02:43:58.575637 29269 leader_election.cc:302] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3080a7113294155897915ae876400c7, cd859eb96e974fba869481bce2d02bb1; no voters:
I20260808 02:43:58.576081 29742 raft_consensus.cc:2781] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:43:58.576305 29742 raft_consensus.cc:686] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 1 LEADER]: Becoming Leader. State: Replica: cd859eb96e974fba869481bce2d02bb1, State: Running, Role: LEADER
I20260808 02:43:58.576622 29742 consensus_queue.cc:237] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:43:58.579962 28932 catalog_manager.cc:5495] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd859eb96e974fba869481bce2d02bb1 (127.25.137.194). New cstate: current_term: 1 leader_uuid: "cd859eb96e974fba869481bce2d02bb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } health_report { overall_health: UNKNOWN } } }
I20260808 02:43:58.620324 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.613752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49962 (local address 127.25.137.194:43933)
0808 02:43:58.613942 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:58.613949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:58.614184 (+ 235us) server_negotiation.cc:407] Connection header received
0808 02:43:58.614396 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:58.614403 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:58.614496 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:58.614606 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:43:58.615012 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.616176 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.618132 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.618866 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.619018 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.619470 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:43:58.619650 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:43:58.619751 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:58.619908 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 02:43:58.619999 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:43:58.638566 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.631300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:43933 (local address 127.25.137.194:58839)
0808 02:43:58.631693 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 02:43:58.631712 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:43:58.631853 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:43:58.632247 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:43:58.632252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:43:58.632266 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:43:58.632491 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.632502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.634028 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.634034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.635757 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:43:58.635774 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.636963 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.636970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:43:58.637215 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.637304 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:43:58.637948 (+ 644us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:43:58.637958 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 02:43:58.638049 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 02:43:58.638198 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253}
I20260808 02:43:58.639365 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:43:58.631365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:58839 (local address 127.25.137.194:43933)
0808 02:43:58.631623 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 02:43:58.631629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:43:58.631794 (+ 165us) server_negotiation.cc:407] Connection header received
0808 02:43:58.632009 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:43:58.632015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:43:58.632104 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:43:58.632205 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:43:58.632653 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.633881 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.635912 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:43:58.636685 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:43:58.636819 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:43:58.637621 (+ 802us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:43:58.637793 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:43:58.637897 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 02:43:58.638939 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0808 02:43:58.639027 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":129}
I20260808 02:43:58.658190 29605 raft_consensus.cc:1261] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Refusing update from remote peer cd859eb96e974fba869481bce2d02bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:43:58.658372 29169 raft_consensus.cc:1261] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Refusing update from remote peer cd859eb96e974fba869481bce2d02bb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:43:58.658876 29742 consensus_queue.cc:1035] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:43:58.661504 29743 consensus_queue.cc:1035] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:43:58.704249 29750 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 02:43:58.727454 29752 mvcc.cc:205] Tried to move back new op lower bound from 7316099230338572288 to 7316099230013493248. Current Snapshot: MvccSnapshot[applied={T|T < 7316099230338572288}]
I20260808 02:44:00.314538 28932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:00.314769 28932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:00.318992 28932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70: Sending DeleteTablet for 3 replicas of tablet 97159a8931c24782b6c1e66c11c8846c
I20260808 02:44:00.319808 29133 tablet_service.cc:1514] Processing DeleteTablet for tablet 97159a8931c24782b6c1e66c11c8846c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:00 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:44926
I20260808 02:44:00.319839 29345 tablet_service.cc:1514] Processing DeleteTablet for tablet 97159a8931c24782b6c1e66c11c8846c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:00 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:49942
I20260808 02:44:00.319926 29562 tablet_service.cc:1514] Processing DeleteTablet for tablet 97159a8931c24782b6c1e66c11c8846c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:00 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:40146
I20260808 02:44:00.335276 29828 tablet_replica.cc:330] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec: stopping tablet replica
I20260808 02:44:00.335518 29828 raft_consensus.cc:2229] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:00.335929 29828 raft_consensus.cc:2258] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:00.336632 29826 tablet_replica.cc:330] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7: stopping tablet replica
I20260808 02:44:00.336917 29826 raft_consensus.cc:2229] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:00.337092 29828 ts_tablet_manager.cc:1905] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:00.337255 29826 raft_consensus.cc:2258] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:00.337950 29825 tablet_replica.cc:330] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1: stopping tablet replica
I20260808 02:44:00.338143 29825 raft_consensus.cc:2229] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:44:00.338820 29825 raft_consensus.cc:2258] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:00.339073 29826 ts_tablet_manager.cc:1905] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:00.339911 29825 ts_tablet_manager.cc:1905] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:00.343613 29828 ts_tablet_manager.cc:1918] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:44:00.343827 29828 log.cc:1196] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/97159a8931c24782b6c1e66c11c8846c
I20260808 02:44:00.344348 29828 ts_tablet_manager.cc:1939] T 97159a8931c24782b6c1e66c11c8846c P 199527ca6fb04c479c200b0fe38b17ec: Deleting consensus metadata
I20260808 02:44:00.345577 29826 ts_tablet_manager.cc:1918] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:44:00.345608 28910 catalog_manager.cc:4841] TS 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901): tablet 97159a8931c24782b6c1e66c11c8846c (table test-table [id=d757fea281f742d481bed78f5891fe2d]) successfully deleted
I20260808 02:44:00.345748 29826 log.cc:1196] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/97159a8931c24782b6c1e66c11c8846c
I20260808 02:44:00.346179 29826 ts_tablet_manager.cc:1939] T 97159a8931c24782b6c1e66c11c8846c P c3080a7113294155897915ae876400c7: Deleting consensus metadata
I20260808 02:44:00.346421 29825 ts_tablet_manager.cc:1918] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:44:00.346555 29825 log.cc:1196] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/97159a8931c24782b6c1e66c11c8846c
I20260808 02:44:00.347029 29825 ts_tablet_manager.cc:1939] T 97159a8931c24782b6c1e66c11c8846c P cd859eb96e974fba869481bce2d02bb1: Deleting consensus metadata
I20260808 02:44:00.347465 28910 catalog_manager.cc:4841] TS c3080a7113294155897915ae876400c7 (127.25.137.193:41301): tablet 97159a8931c24782b6c1e66c11c8846c (table test-table [id=d757fea281f742d481bed78f5891fe2d]) successfully deleted
I20260808 02:44:00.348055 28911 catalog_manager.cc:4841] TS cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933): tablet 97159a8931c24782b6c1e66c11c8846c (table test-table [id=d757fea281f742d481bed78f5891fe2d]) successfully deleted
Aug 08 02:44:00 dist-test-slave-k5rr krb5kdc[28865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:44:00 dist-test-slave-k5rr krb5kdc[28865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:00.387277 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:00.373221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.25.137.254:42093)
0808 02:44:00.373777 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:00.373784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:00.373855 (+ 71us) server_negotiation.cc:407] Connection header received
0808 02:44:00.374059 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:00.374064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:00.374129 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:00.374220 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:00.375548 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:00.376375 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:00.377669 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:00.378090 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:00.378219 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:00.382000 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:00.382005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:00.382008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:00.382042 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:00.384736 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:00.385397 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:00.385401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:00.385403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:00.385526 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:00.385895 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:00.385899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:00.385901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:00.386173 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:00.386412 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:00.386696 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 02:44:00.386906 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":248,"threads_started":1}
I20260808 02:44:00.390713 28932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:44:00.391376 28932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:44:00.398208 29133 tablet_service.cc:1467] Processing CreateTablet for tablet 183416aa79b0443c9c868b5d70454b5d (DEFAULT_TABLE table=test-table [id=dec2f2f39e98423b99b72f2d9911cd03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:00.398491 29345 tablet_service.cc:1467] Processing CreateTablet for tablet 183416aa79b0443c9c868b5d70454b5d (DEFAULT_TABLE table=test-table [id=dec2f2f39e98423b99b72f2d9911cd03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:00.398682 29133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 183416aa79b0443c9c868b5d70454b5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:00.398619 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 183416aa79b0443c9c868b5d70454b5d (DEFAULT_TABLE table=test-table [id=dec2f2f39e98423b99b72f2d9911cd03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:00.398887 29345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 183416aa79b0443c9c868b5d70454b5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:00.399075 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 183416aa79b0443c9c868b5d70454b5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:00.404893 29841 tablet_bootstrap.cc:492] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1: Bootstrap starting.
I20260808 02:44:00.405576 29843 tablet_bootstrap.cc:492] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7: Bootstrap starting.
I20260808 02:44:00.406599 29841 tablet_bootstrap.cc:654] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:00.407310 29843 tablet_bootstrap.cc:654] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:00.409704 29844 tablet_bootstrap.cc:492] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec: Bootstrap starting.
I20260808 02:44:00.409729 29843 tablet_bootstrap.cc:492] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7: No bootstrap required, opened a new log
I20260808 02:44:00.409870 29843 ts_tablet_manager.cc:1397] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260808 02:44:00.410681 29843 raft_consensus.cc:348] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.410915 29843 raft_consensus.cc:374] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:00.411026 29843 raft_consensus.cc:729] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3080a7113294155897915ae876400c7, State: Initialized, Role: FOLLOWER
I20260808 02:44:00.411339 29844 tablet_bootstrap.cc:654] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:00.411281 29843 consensus_queue.cc:260] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.412386 29843 ts_tablet_manager.cc:1428] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:44:00.412931 29841 tablet_bootstrap.cc:492] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1: No bootstrap required, opened a new log
I20260808 02:44:00.413091 29841 ts_tablet_manager.cc:1397] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20260808 02:44:00.413703 29841 raft_consensus.cc:348] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.413905 29841 raft_consensus.cc:374] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:00.414000 29841 raft_consensus.cc:729] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd859eb96e974fba869481bce2d02bb1, State: Initialized, Role: FOLLOWER
I20260808 02:44:00.414266 29841 consensus_queue.cc:260] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.414872 29841 ts_tablet_manager.cc:1428] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:44:00.416110 29844 tablet_bootstrap.cc:492] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec: No bootstrap required, opened a new log
I20260808 02:44:00.416338 29844 ts_tablet_manager.cc:1397] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 02:44:00.417182 29844 raft_consensus.cc:348] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.417430 29844 raft_consensus.cc:374] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:00.417527 29844 raft_consensus.cc:729] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 199527ca6fb04c479c200b0fe38b17ec, State: Initialized, Role: FOLLOWER
I20260808 02:44:00.417757 29844 consensus_queue.cc:260] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.418497 29844 ts_tablet_manager.cc:1428] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:44:00.685909 29850 raft_consensus.cc:482] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:00.686169 29850 raft_consensus.cc:504] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.686937 29850 leader_election.cc:288] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3080a7113294155897915ae876400c7 (127.25.137.193:41301), cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933)
I20260808 02:44:00.687631 29372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183416aa79b0443c9c868b5d70454b5d" candidate_uuid: "199527ca6fb04c479c200b0fe38b17ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd859eb96e974fba869481bce2d02bb1" is_pre_election: true
I20260808 02:44:00.687621 29169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183416aa79b0443c9c868b5d70454b5d" candidate_uuid: "199527ca6fb04c479c200b0fe38b17ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3080a7113294155897915ae876400c7" is_pre_election: true
I20260808 02:44:00.687891 29372 raft_consensus.cc:2454] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 199527ca6fb04c479c200b0fe38b17ec in term 0.
I20260808 02:44:00.687956 29169 raft_consensus.cc:2454] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 199527ca6fb04c479c200b0fe38b17ec in term 0.
I20260808 02:44:00.688400 29489 leader_election.cc:302] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 199527ca6fb04c479c200b0fe38b17ec, cd859eb96e974fba869481bce2d02bb1; no voters:
I20260808 02:44:00.688673 29850 raft_consensus.cc:2781] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:44:00.688796 29850 raft_consensus.cc:482] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:44:00.688892 29850 raft_consensus.cc:3037] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:00.691210 29850 raft_consensus.cc:504] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.691867 29850 leader_election.cc:288] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [CANDIDATE]: Term 1 election: Requested vote from peers c3080a7113294155897915ae876400c7 (127.25.137.193:41301), cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933)
I20260808 02:44:00.692363 29169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183416aa79b0443c9c868b5d70454b5d" candidate_uuid: "199527ca6fb04c479c200b0fe38b17ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3080a7113294155897915ae876400c7"
I20260808 02:44:00.692399 29372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183416aa79b0443c9c868b5d70454b5d" candidate_uuid: "199527ca6fb04c479c200b0fe38b17ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd859eb96e974fba869481bce2d02bb1"
I20260808 02:44:00.692606 29169 raft_consensus.cc:3037] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:00.692618 29372 raft_consensus.cc:3037] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:00.694852 29372 raft_consensus.cc:2454] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 199527ca6fb04c479c200b0fe38b17ec in term 1.
I20260808 02:44:00.694850 29169 raft_consensus.cc:2454] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 199527ca6fb04c479c200b0fe38b17ec in term 1.
I20260808 02:44:00.695274 29488 leader_election.cc:302] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 199527ca6fb04c479c200b0fe38b17ec, c3080a7113294155897915ae876400c7; no voters:
I20260808 02:44:00.695600 29850 raft_consensus.cc:2781] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:00.696144 29850 raft_consensus.cc:686] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 1 LEADER]: Becoming Leader. State: Replica: 199527ca6fb04c479c200b0fe38b17ec, State: Running, Role: LEADER
I20260808 02:44:00.696512 29850 consensus_queue.cc:237] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.699168 28930 catalog_manager.cc:5495] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195). New cstate: current_term: 1 leader_uuid: "199527ca6fb04c479c200b0fe38b17ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } health_report { overall_health: HEALTHY } } }
I20260808 02:44:00.781628 29856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:00.776445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40160 (local address 127.25.137.195:37901)
0808 02:44:00.777081 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:00.777087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:00.777107 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:44:00.777173 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:00.777177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:00.777252 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:00.777372 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:00.777766 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:00.778526 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:00.779858 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:00.780290 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:00.780409 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:00.780874 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:00.780975 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:00.781095 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:00.781230 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 02:44:00.781316 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":321,"threads_started":1}
I20260808 02:44:00.790330 29372 raft_consensus.cc:1261] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Refusing update from remote peer 199527ca6fb04c479c200b0fe38b17ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:00.790870 29169 raft_consensus.cc:1261] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Refusing update from remote peer 199527ca6fb04c479c200b0fe38b17ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:00.790983 29851 consensus_queue.cc:1035] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:00.791414 29850 consensus_queue.cc:1035] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:00.795370 29802 mvcc.cc:205] Tried to move back new op lower bound from 7316099239072239616 to 7316099238695448576. Current Snapshot: MvccSnapshot[applied={T|T < 7316099239072239616}]
I20260808 02:44:00.836112 28930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:00.836390 28930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:00.839340 28930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70: Sending DeleteTablet for 3 replicas of tablet 183416aa79b0443c9c868b5d70454b5d
I20260808 02:44:00.840158 29133 tablet_service.cc:1514] Processing DeleteTablet for tablet 183416aa79b0443c9c868b5d70454b5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:00 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:44926
I20260808 02:44:00.840196 29345 tablet_service.cc:1514] Processing DeleteTablet for tablet 183416aa79b0443c9c868b5d70454b5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:00 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:49942
I20260808 02:44:00.840471 29826 tablet_replica.cc:330] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7: stopping tablet replica
I20260808 02:44:00.840485 29825 tablet_replica.cc:330] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1: stopping tablet replica
I20260808 02:44:00.840674 29826 raft_consensus.cc:2229] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:00.840703 29825 raft_consensus.cc:2229] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:00.840868 29826 raft_consensus.cc:2258] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:00.840945 29825 raft_consensus.cc:2258] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:00.841176 29562 tablet_service.cc:1514] Processing DeleteTablet for tablet 183416aa79b0443c9c868b5d70454b5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:00 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:40146
I20260808 02:44:00.841454 29828 tablet_replica.cc:330] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec: stopping tablet replica
I20260808 02:44:00.841648 29828 raft_consensus.cc:2229] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:44:00.841707 28930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 02:44:00.841957 29828 raft_consensus.cc:2258] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:00.842272 29825 ts_tablet_manager.cc:1905] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:44:00.842370 28930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:44:00.842496 29826 ts_tablet_manager.cc:1905] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:00.843480 29828 ts_tablet_manager.cc:1905] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:00.848199 29825 ts_tablet_manager.cc:1918] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:44:00.848392 29825 log.cc:1196] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/183416aa79b0443c9c868b5d70454b5d
I20260808 02:44:00.848882 29825 ts_tablet_manager.cc:1939] T 183416aa79b0443c9c868b5d70454b5d P cd859eb96e974fba869481bce2d02bb1: Deleting consensus metadata
I20260808 02:44:00.849468 29826 ts_tablet_manager.cc:1918] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:44:00.849639 29826 log.cc:1196] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/183416aa79b0443c9c868b5d70454b5d
I20260808 02:44:00.850111 29826 ts_tablet_manager.cc:1939] T 183416aa79b0443c9c868b5d70454b5d P c3080a7113294155897915ae876400c7: Deleting consensus metadata
I20260808 02:44:00.850540 28911 catalog_manager.cc:4841] TS cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933): tablet 183416aa79b0443c9c868b5d70454b5d (table test-table [id=dec2f2f39e98423b99b72f2d9911cd03]) successfully deleted
I20260808 02:44:00.851405 28910 catalog_manager.cc:4841] TS c3080a7113294155897915ae876400c7 (127.25.137.193:41301): tablet 183416aa79b0443c9c868b5d70454b5d (table test-table [id=dec2f2f39e98423b99b72f2d9911cd03]) successfully deleted
I20260808 02:44:00.852241 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 636e422c8f4845b9b8903917cc5ed647 (DEFAULT_TABLE table=test-table [id=c36dadd92aa949c7bdcfda0d52ad1318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:00.852272 29133 tablet_service.cc:1467] Processing CreateTablet for tablet 636e422c8f4845b9b8903917cc5ed647 (DEFAULT_TABLE table=test-table [id=c36dadd92aa949c7bdcfda0d52ad1318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:00.852366 29345 tablet_service.cc:1467] Processing CreateTablet for tablet 636e422c8f4845b9b8903917cc5ed647 (DEFAULT_TABLE table=test-table [id=c36dadd92aa949c7bdcfda0d52ad1318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:00.852619 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 636e422c8f4845b9b8903917cc5ed647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:00.852712 29133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 636e422c8f4845b9b8903917cc5ed647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:00.852761 29345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 636e422c8f4845b9b8903917cc5ed647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:00.857102 29844 tablet_bootstrap.cc:492] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec: Bootstrap starting.
I20260808 02:44:00.858569 29828 ts_tablet_manager.cc:1918] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:44:00.858722 29828 log.cc:1196] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/183416aa79b0443c9c868b5d70454b5d
I20260808 02:44:00.858814 29844 tablet_bootstrap.cc:654] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:00.859148 29841 tablet_bootstrap.cc:492] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1: Bootstrap starting.
I20260808 02:44:00.859839 29843 tablet_bootstrap.cc:492] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7: Bootstrap starting.
I20260808 02:44:00.860674 29841 tablet_bootstrap.cc:654] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:00.861313 29828 ts_tablet_manager.cc:1939] T 183416aa79b0443c9c868b5d70454b5d P 199527ca6fb04c479c200b0fe38b17ec: Deleting consensus metadata
I20260808 02:44:00.861419 29843 tablet_bootstrap.cc:654] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:00.862421 28910 catalog_manager.cc:4841] TS 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901): tablet 183416aa79b0443c9c868b5d70454b5d (table test-table [id=dec2f2f39e98423b99b72f2d9911cd03]) successfully deleted
I20260808 02:44:00.863366 29844 tablet_bootstrap.cc:492] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec: No bootstrap required, opened a new log
I20260808 02:44:00.863548 29844 ts_tablet_manager.cc:1397] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260808 02:44:00.864320 29844 raft_consensus.cc:348] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.864601 29844 raft_consensus.cc:374] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:00.864688 29844 raft_consensus.cc:729] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 199527ca6fb04c479c200b0fe38b17ec, State: Initialized, Role: FOLLOWER
I20260808 02:44:00.864949 29844 consensus_queue.cc:260] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.865614 29841 tablet_bootstrap.cc:492] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1: No bootstrap required, opened a new log
I20260808 02:44:00.865777 29841 ts_tablet_manager.cc:1397] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260808 02:44:00.866354 29843 tablet_bootstrap.cc:492] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7: No bootstrap required, opened a new log
I20260808 02:44:00.866523 29843 ts_tablet_manager.cc:1397] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260808 02:44:00.866542 29841 raft_consensus.cc:348] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.866806 29841 raft_consensus.cc:374] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:00.866894 29841 raft_consensus.cc:729] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd859eb96e974fba869481bce2d02bb1, State: Initialized, Role: FOLLOWER
I20260808 02:44:00.867123 29841 consensus_queue.cc:260] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.867237 29843 raft_consensus.cc:348] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.867507 29843 raft_consensus.cc:374] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:00.867594 29843 raft_consensus.cc:729] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3080a7113294155897915ae876400c7, State: Initialized, Role: FOLLOWER
I20260808 02:44:00.867682 29841 ts_tablet_manager.cc:1428] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 02:44:00.867825 29843 consensus_queue.cc:260] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:00.868474 29843 ts_tablet_manager.cc:1428] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260808 02:44:00.868934 29844 ts_tablet_manager.cc:1428] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260808 02:44:01.018421 29847 raft_consensus.cc:482] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:01.018636 29847 raft_consensus.cc:504] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:01.019337 29847 leader_election.cc:288] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933), 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901)
I20260808 02:44:01.019958 29372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636e422c8f4845b9b8903917cc5ed647" candidate_uuid: "c3080a7113294155897915ae876400c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd859eb96e974fba869481bce2d02bb1" is_pre_election: true
I20260808 02:44:01.019996 29605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636e422c8f4845b9b8903917cc5ed647" candidate_uuid: "c3080a7113294155897915ae876400c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "199527ca6fb04c479c200b0fe38b17ec" is_pre_election: true
I20260808 02:44:01.020237 29605 raft_consensus.cc:2454] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3080a7113294155897915ae876400c7 in term 0.
I20260808 02:44:01.020238 29372 raft_consensus.cc:2454] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3080a7113294155897915ae876400c7 in term 0.
I20260808 02:44:01.020720 29060 leader_election.cc:302] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 199527ca6fb04c479c200b0fe38b17ec, c3080a7113294155897915ae876400c7, cd859eb96e974fba869481bce2d02bb1; no voters:
I20260808 02:44:01.021066 29847 raft_consensus.cc:2781] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:44:01.021178 29847 raft_consensus.cc:482] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:44:01.021270 29847 raft_consensus.cc:3037] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:01.023488 29847 raft_consensus.cc:504] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:01.024098 29847 leader_election.cc:288] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [CANDIDATE]: Term 1 election: Requested vote from peers cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933), 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901)
I20260808 02:44:01.024700 29605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636e422c8f4845b9b8903917cc5ed647" candidate_uuid: "c3080a7113294155897915ae876400c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "199527ca6fb04c479c200b0fe38b17ec"
I20260808 02:44:01.024700 29372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636e422c8f4845b9b8903917cc5ed647" candidate_uuid: "c3080a7113294155897915ae876400c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd859eb96e974fba869481bce2d02bb1"
I20260808 02:44:01.024931 29605 raft_consensus.cc:3037] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:01.024930 29372 raft_consensus.cc:3037] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:01.027951 29372 raft_consensus.cc:2454] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3080a7113294155897915ae876400c7 in term 1.
I20260808 02:44:01.027948 29605 raft_consensus.cc:2454] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3080a7113294155897915ae876400c7 in term 1.
I20260808 02:44:01.028481 29060 leader_election.cc:302] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 199527ca6fb04c479c200b0fe38b17ec, c3080a7113294155897915ae876400c7, cd859eb96e974fba869481bce2d02bb1; no voters:
I20260808 02:44:01.028780 29847 raft_consensus.cc:2781] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:01.029230 29847 raft_consensus.cc:686] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 1 LEADER]: Becoming Leader. State: Replica: c3080a7113294155897915ae876400c7, State: Running, Role: LEADER
I20260808 02:44:01.029628 29847 consensus_queue.cc:237] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } }
I20260808 02:44:01.032940 28932 catalog_manager.cc:5495] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3080a7113294155897915ae876400c7 (127.25.137.193). New cstate: current_term: 1 leader_uuid: "c3080a7113294155897915ae876400c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3080a7113294155897915ae876400c7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 } health_report { overall_health: UNKNOWN } } }
I20260808 02:44:01.068457 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:01.062907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44932 (local address 127.25.137.193:41301)
0808 02:44:01.063506 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:01.063510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:01.063529 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:44:01.063611 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:01.063615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:01.063685 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:01.063825 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:01.064269 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:01.065158 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:01.066382 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:01.066807 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:01.066895 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:01.067263 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:01.067847 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:01.067953 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:01.068052 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 02:44:01.068124 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":334,"threads_started":1}
I20260808 02:44:01.083815 29878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:01.078429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:58155 (local address 127.25.137.194:43933)
0808 02:44:01.078993 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:01.078998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:01.079164 (+ 166us) server_negotiation.cc:407] Connection header received
0808 02:44:01.079343 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:01.079346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:01.079410 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:01.079535 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:01.079978 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:01.080748 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:01.082186 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:01.082655 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:01.082748 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:01.083153 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:01.083260 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:01.083344 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:01.083431 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 02:44:01.083494 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":308,"threads_started":1}
I20260808 02:44:01.083809 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:01.078298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:43933 (local address 127.25.137.193:58155)
0808 02:44:01.078937 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:01.078953 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:01.079229 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:01.079557 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:01.079560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:01.079569 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:44:01.079823 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:01.079831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:01.080870 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:01.080874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:01.082048 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:01.082060 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:01.082801 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:01.082806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:01.082942 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:01.082994 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:44:01.083341 (+ 347us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:44:01.083347 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:44:01.083398 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 02:44:01.083503 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":300,"threads_started":1}
I20260808 02:44:01.092360 29605 raft_consensus.cc:1261] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Refusing update from remote peer c3080a7113294155897915ae876400c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:01.092389 29372 raft_consensus.cc:1261] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Refusing update from remote peer c3080a7113294155897915ae876400c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:01.092960 29870 consensus_queue.cc:1035] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "199527ca6fb04c479c200b0fe38b17ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 37901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:01.093425 29847 consensus_queue.cc:1035] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd859eb96e974fba869481bce2d02bb1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 43933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:01.121743 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:01.114925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:46681 (local address 127.25.137.193:41301)
0808 02:44:01.115119 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:01.115126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:01.116728 (+ 1602us) server_negotiation.cc:407] Connection header received
0808 02:44:01.116956 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:01.116962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:01.117069 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:01.117162 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:01.117663 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:01.118598 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:01.120088 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:01.120492 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:01.120584 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:01.120997 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:01.121138 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:01.121205 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:01.121409 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 02:44:01.121494 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:44:01.121807 29886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:01.114729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:41301 (local address 127.25.137.194:46681)
0808 02:44:01.116472 (+ 1743us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:01.116492 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:01.116810 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:01.117180 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:01.117184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:01.117199 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:44:01.117499 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:01.117509 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:01.118733 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:01.118739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:01.119944 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:01.119958 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:01.120668 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:01.120673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:01.120812 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:01.120851 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:44:01.121225 (+ 374us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:44:01.121238 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 02:44:01.121333 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 02:44:01.121451 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1580,"thread_start_us":1457,"threads_started":1}
W20260808 02:44:01.123099 29804 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 02:44:02.733693 28932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:02.733946 28932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:02.738193 28932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c2c437417614743865926995c658d70: Sending DeleteTablet for 3 replicas of tablet 636e422c8f4845b9b8903917cc5ed647
I20260808 02:44:02.738940 29133 tablet_service.cc:1514] Processing DeleteTablet for tablet 636e422c8f4845b9b8903917cc5ed647 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:02 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:44926
I20260808 02:44:02.739089 29345 tablet_service.cc:1514] Processing DeleteTablet for tablet 636e422c8f4845b9b8903917cc5ed647 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:02 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:49942
I20260808 02:44:02.739110 29562 tablet_service.cc:1514] Processing DeleteTablet for tablet 636e422c8f4845b9b8903917cc5ed647 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:02 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:40146
I20260808 02:44:02.739635 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29034
I20260808 02:44:02.751655 29920 tablet_replica.cc:330] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7: stopping tablet replica
I20260808 02:44:02.752079 29920 raft_consensus.cc:2229] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:44:02.752774 29920 raft_consensus.cc:2258] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:02.754056 29920 ts_tablet_manager.cc:1905] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:02.754415 29921 tablet_replica.cc:330] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1: stopping tablet replica
I20260808 02:44:02.754767 29921 raft_consensus.cc:2229] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:02.755119 29921 raft_consensus.cc:2258] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:02.755318 29922 tablet_replica.cc:330] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec: stopping tablet replica
I20260808 02:44:02.755622 29922 raft_consensus.cc:2229] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:02.755923 29922 raft_consensus.cc:2258] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:02.756222 29921 ts_tablet_manager.cc:1905] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:02.757360 29922 ts_tablet_manager.cc:1905] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:02.761497 29920 ts_tablet_manager.cc:1918] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:44:02.761631 29920 log.cc:1196] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/636e422c8f4845b9b8903917cc5ed647
I20260808 02:44:02.762145 29920 ts_tablet_manager.cc:1939] T 636e422c8f4845b9b8903917cc5ed647 P c3080a7113294155897915ae876400c7: Deleting consensus metadata
I20260808 02:44:02.762779 29921 ts_tablet_manager.cc:1918] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:44:02.763064 29921 log.cc:1196] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/636e422c8f4845b9b8903917cc5ed647
I20260808 02:44:02.763644 29921 ts_tablet_manager.cc:1939] T 636e422c8f4845b9b8903917cc5ed647 P cd859eb96e974fba869481bce2d02bb1: Deleting consensus metadata
I20260808 02:44:02.763643 29922 ts_tablet_manager.cc:1918] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:44:02.763645 28910 catalog_manager.cc:4841] TS c3080a7113294155897915ae876400c7 (127.25.137.193:41301): tablet 636e422c8f4845b9b8903917cc5ed647 (table test-table [id=c36dadd92aa949c7bdcfda0d52ad1318]) successfully deleted
I20260808 02:44:02.763779 29922 log.cc:1196] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/636e422c8f4845b9b8903917cc5ed647
I20260808 02:44:02.764191 29922 ts_tablet_manager.cc:1939] T 636e422c8f4845b9b8903917cc5ed647 P 199527ca6fb04c479c200b0fe38b17ec: Deleting consensus metadata
Aug 08 02:44:02 dist-test-slave-k5rr krb5kdc[28865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:44:02.764812 28911 catalog_manager.cc:4841] TS cd859eb96e974fba869481bce2d02bb1 (127.25.137.194:43933): tablet 636e422c8f4845b9b8903917cc5ed647 (table test-table [id=c36dadd92aa949c7bdcfda0d52ad1318]) successfully deleted
I20260808 02:44:02.765272 28910 catalog_manager.cc:4841] TS 199527ca6fb04c479c200b0fe38b17ec (127.25.137.195:37901): tablet 636e422c8f4845b9b8903917cc5ed647 (table test-table [id=c36dadd92aa949c7bdcfda0d52ad1318]) successfully deleted
I20260808 02:44:02.771170 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:02.742058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40776 (local address 127.25.137.193:41301)
0808 02:44:02.748159 (+ 6101us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:02.748166 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:02.753835 (+ 5669us) server_negotiation.cc:407] Connection header received
0808 02:44:02.754353 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:02.754359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:02.754440 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:02.754525 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:02.756397 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:02.757567 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:02.759442 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:02.760130 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:02.760226 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:02.765969 (+ 5743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:02.765990 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:02.766011 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:02.766060 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:02.768914 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:02.769540 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:02.769547 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:02.769549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:02.769676 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:02.769945 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:02.769948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:02.769950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:02.770307 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:02.770528 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:02.770720 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 02:44:02.770895 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5929,"thread_start_us":5769,"threads_started":1}
W20260808 02:44:02.771874 29227 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40776
W20260808 02:44:02.772490 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29034: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:44:02.772689 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 29034
I20260808 02:44:02.804035 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29240
Aug 08 02:44:02 dist-test-slave-k5rr krb5kdc[28865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:44:02.818904 29926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:02.804819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46870 (local address 127.25.137.194:43933)
0808 02:44:02.805350 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:02.805355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:02.805372 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:44:02.805427 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:02.805431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:02.805496 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:02.805588 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:02.806847 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:02.807602 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:02.808845 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:02.809323 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:02.809421 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:02.813355 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:02.813369 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:02.813386 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:02.813418 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:02.816340 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:02.816916 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:02.816924 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:02.816926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:02.817218 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:02.817593 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:02.817596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:02.817598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:02.817932 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:02.818176 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:02.818387 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 02:44:02.818548 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":296,"threads_started":1}
W20260808 02:44:02.819554 29445 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46870
W20260808 02:44:02.820165 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29240: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:44:02.820334 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 29240
I20260808 02:44:02.853672 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29459
Aug 08 02:44:02 dist-test-slave-k5rr krb5kdc[28865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:44:02.868470 29928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:02.854402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38980 (local address 127.25.137.195:37901)
0808 02:44:02.854898 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:02.854903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:02.854920 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:44:02.854975 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:02.854978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:02.855036 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:02.855127 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:02.856405 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:02.857254 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:02.858447 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:02.858868 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:02.858961 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:02.863057 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:02.863074 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:02.863089 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:02.863134 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:02.865775 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:02.866453 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:02.866460 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:02.866463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:02.866609 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:02.866966 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:02.866970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:02.866972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:02.867339 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:02.867616 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:02.867917 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 02:44:02.868141 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":264,"threads_started":1}
W20260808 02:44:02.869168 29663 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38980
W20260808 02:44:02.869789 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29459: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:44:02.869930 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 29459
I20260808 02:44:02.897500 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28884
I20260808 02:44:02.898723 28993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:03.016537 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 28884
2026-08-08T02:44:03Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (10150 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:44:03 dist-test-slave-k5rr krb5kdc[29935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:44:03 dist-test-slave-k5rr krb5kdc[29935](info): set up 2 sockets
Aug 08 02:44:03 dist-test-slave-k5rr krb5kdc[29935](info): commencing operation
krb5kdc: starting...
W20260808 02:44:04.220700 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:44:04 dist-test-slave-k5rr krb5kdc[29935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:44:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:44:04.430354 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:37801
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:37801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:04.759178 29954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:04.759498 29954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:04.759645 29954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:04.766971 29954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:04.767100 29954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:04.767186 29954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:04.767271 29954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:04.767339 29954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:04.777470 29954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:37801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:37801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:04.778836 29954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:04.780196 29954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:04.792290 29969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:04.792380 29965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:04.792399 29966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:04.793315 29954 server_base.cc:1034] running on GCE node
I20260808 02:44:04.794169 29954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:04.795639 29954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:04.796890 29954 hybrid_clock.cc:648] HybridClock initialized: now 1786157044796801 us; error 68 us; skew 500 ppm
I20260808 02:44:04.797288 29954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:04 dist-test-slave-k5rr krb5kdc[29935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157044, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:04.800465 29954 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:44:04.802057 29954 webserver.cc:460] Webserver started at http://127.25.137.254:38257/ using document root <none> and password file <none>
I20260808 02:44:04.802745 29954 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:04.802876 29954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:04.803188 29954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:04.806278 29954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "c535c7e9fe614e50a2ab1e15d201f5b8"
format_stamp: "Formatted at 2026-08-08 02:44:04 on dist-test-slave-k5rr"
server_key: "1bc93e9db229a975f63024dee7f1ee33"
server_key_iv: "03d1caf401bd1a47c755d644e3087c67"
server_key_version: "encryptionkey@0"
I20260808 02:44:04.807044 29954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "c535c7e9fe614e50a2ab1e15d201f5b8"
format_stamp: "Formatted at 2026-08-08 02:44:04 on dist-test-slave-k5rr"
server_key: "1bc93e9db229a975f63024dee7f1ee33"
server_key_iv: "03d1caf401bd1a47c755d644e3087c67"
server_key_version: "encryptionkey@0"
I20260808 02:44:04.812245 29954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 02:44:04.815801 29977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:04.817199 29954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:44:04.817369 29954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "c535c7e9fe614e50a2ab1e15d201f5b8"
format_stamp: "Formatted at 2026-08-08 02:44:04 on dist-test-slave-k5rr"
server_key: "1bc93e9db229a975f63024dee7f1ee33"
server_key_iv: "03d1caf401bd1a47c755d644e3087c67"
server_key_version: "encryptionkey@0"
I20260808 02:44:04.817553 29954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:04.830196 29954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:04.834410 29954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:04.834699 29954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:04.854050 29954 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:37801
I20260808 02:44:04.854068 30040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:37801 every 8 connection(s)
I20260808 02:44:04.855582 29954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:04.858538 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 29954
I20260808 02:44:04.858753 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:04.859575 26151 external_mini_cluster.cc:1428] Setting key 31e314b79803835fdc1a0ef4cddbc419
I20260808 02:44:04.860235 30041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:04.874635 30041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8: Bootstrap starting.
Aug 08 02:44:04 dist-test-slave-k5rr krb5kdc[29935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:04.878005 30041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:04.879066 30041 log.cc:824] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:04.882030 30041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8: No bootstrap required, opened a new log
I20260808 02:44:04.883701 30045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:04.861695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58932 (local address 127.25.137.254:37801)
0808 02:44:04.862417 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:04.862428 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:04.862469 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:44:04.864881 (+ 2412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:04.864906 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:04.865345 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:04.865662 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:04.867468 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:04.868816 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:04.870599 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:04.871353 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:04.871500 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:04.876017 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:04.876025 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:04.876045 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:04.876089 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:04.879327 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:04.880168 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:04.880175 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:04.880181 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:04.880384 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:04.880810 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:04.880815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:04.880817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:04.881430 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:04.881727 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:04.882212 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 02:44:04.882646 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":319,"threads_started":1}
I20260808 02:44:04.889750 30041 raft_consensus.cc:348] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c535c7e9fe614e50a2ab1e15d201f5b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37801 } }
I20260808 02:44:04.890108 30041 raft_consensus.cc:374] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:04.890211 30041 raft_consensus.cc:729] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c535c7e9fe614e50a2ab1e15d201f5b8, State: Initialized, Role: FOLLOWER
I20260808 02:44:04.890738 30041 consensus_queue.cc:260] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c535c7e9fe614e50a2ab1e15d201f5b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37801 } }
I20260808 02:44:04.890976 30041 raft_consensus.cc:388] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:04.891101 30041 raft_consensus.cc:482] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:04.891258 30041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:04.893850 30041 raft_consensus.cc:504] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c535c7e9fe614e50a2ab1e15d201f5b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37801 } }
I20260808 02:44:04.894353 30041 leader_election.cc:302] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c535c7e9fe614e50a2ab1e15d201f5b8; no voters:
I20260808 02:44:04.894896 30041 leader_election.cc:288] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:04.895030 30047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:04.895676 30047 raft_consensus.cc:686] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [term 1 LEADER]: Becoming Leader. State: Replica: c535c7e9fe614e50a2ab1e15d201f5b8, State: Running, Role: LEADER
I20260808 02:44:04.896255 30047 consensus_queue.cc:237] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c535c7e9fe614e50a2ab1e15d201f5b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37801 } }
I20260808 02:44:04.896884 30041 sys_catalog.cc:564] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:04.898888 30050 sys_catalog.cc:455] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c535c7e9fe614e50a2ab1e15d201f5b8. Latest consensus state: current_term: 1 leader_uuid: "c535c7e9fe614e50a2ab1e15d201f5b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c535c7e9fe614e50a2ab1e15d201f5b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37801 } } }
I20260808 02:44:04.899273 30050 sys_catalog.cc:458] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:04.900128 30049 sys_catalog.cc:455] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c535c7e9fe614e50a2ab1e15d201f5b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c535c7e9fe614e50a2ab1e15d201f5b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37801 } } }
I20260808 02:44:04.900503 30049 sys_catalog.cc:458] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:04.901925 30058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:04.907568 30058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:04.916540 30058 catalog_manager.cc:1347] Generated new cluster ID: c13ce4edf0294960b03f93bd4a19ed6e
I20260808 02:44:04.916661 30058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:04.954264 30058 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:04.956199 30058 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:04.964385 30058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c535c7e9fe614e50a2ab1e15d201f5b8: Generated new TSK 0
I20260808 02:44:04.965507 30058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:44:05.020722 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37801
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:05.354395 30078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:05.354661 30078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:05.354794 30078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:05.362049 30078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:05.362154 30078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:05.362304 30078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:44:05.371953 30078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:05.373238 30078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:05.374552 30078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:05.387471 30086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:05.387434 30087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:05.387742 30094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:05.388374 30078 server_base.cc:1034] running on GCE node
I20260808 02:44:05.388959 30078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:05.389942 30078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:05.391160 30078 hybrid_clock.cc:648] HybridClock initialized: now 1786157045391098 us; error 53 us; skew 500 ppm
I20260808 02:44:05.391517 30078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:05 dist-test-slave-k5rr krb5kdc[29935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157045, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:05.394699 30078 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:44:05.396195 30078 webserver.cc:460] Webserver started at http://127.25.137.193:36869/ using document root <none> and password file <none>
I20260808 02:44:05.396896 30078 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:05.397032 30078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:05.397339 30078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:05.400348 30078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "bfa2e39d9a3644698a4137ad08e90b3a"
format_stamp: "Formatted at 2026-08-08 02:44:05 on dist-test-slave-k5rr"
server_key: "2df03666ab46949b2ee6a457ff795741"
server_key_iv: "d1b9f1984ca5172a037a80e7d4d2beb0"
server_key_version: "encryptionkey@0"
I20260808 02:44:05.401276 30078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "bfa2e39d9a3644698a4137ad08e90b3a"
format_stamp: "Formatted at 2026-08-08 02:44:05 on dist-test-slave-k5rr"
server_key: "2df03666ab46949b2ee6a457ff795741"
server_key_iv: "d1b9f1984ca5172a037a80e7d4d2beb0"
server_key_version: "encryptionkey@0"
I20260808 02:44:05.406731 30078 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:44:05.410506 30103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:05.412092 30078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 02:44:05.412261 30078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "bfa2e39d9a3644698a4137ad08e90b3a"
format_stamp: "Formatted at 2026-08-08 02:44:05 on dist-test-slave-k5rr"
server_key: "2df03666ab46949b2ee6a457ff795741"
server_key_iv: "d1b9f1984ca5172a037a80e7d4d2beb0"
server_key_version: "encryptionkey@0"
I20260808 02:44:05.412499 30078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:05.439400 30078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:05.443735 30078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:05.444025 30078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:05.445065 30078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:05.446486 30078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:05.446566 30078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:05.446693 30078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:05.446748 30078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:05.480111 30078 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:43653
I20260808 02:44:05.480151 30262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:43653 every 8 connection(s)
I20260808 02:44:05.481849 30078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:44:05.483418 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 30078
I20260808 02:44:05.483596 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:44:05.484239 26151 external_mini_cluster.cc:1428] Setting key 07da1c4c816cbeb104cc8e7dd5537d6b
Aug 08 02:44:05 dist-test-slave-k5rr krb5kdc[29935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157045, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:05.505817 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:05.484580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:52645 (local address 127.25.137.254:37801)
0808 02:44:05.485067 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:05.485073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:05.489655 (+ 4582us) server_negotiation.cc:407] Connection header received
0808 02:44:05.491559 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:05.491565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:05.491646 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:05.491741 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:05.493229 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:05.493937 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:05.495917 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:05.496314 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:05.496419 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:05.500196 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:05.500201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:05.500204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:05.500236 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:05.502805 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:05.503541 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:05.503545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:05.503547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:05.503668 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:05.504098 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:05.504102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:05.504104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:05.504374 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:05.504515 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:05.505230 (+ 715us) server_negotiation.cc:299] Negotiation successful
0808 02:44:05.505469 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":213,"threads_started":1}
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
I20260808 02:44:05.506723 30267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:05.485172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:37801 (local address 127.25.137.193:52645)
0808 02:44:05.489516 (+ 4344us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:05.489555 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:05.491295 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:05.491979 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:05.491987 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:05.492485 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:05.493068 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:05.493089 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:05.494087 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:05.494091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:05.495764 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:05.495788 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:05.496534 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:05.496541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:05.496820 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:05.497534 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:05.497561 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:05.499991 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:05.502935 (+ 2944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:05.502944 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:05.502960 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:05.503417 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:05.503770 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:05.503776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:05.503780 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:05.503982 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:05.504510 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:05.504518 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:05.504833 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 02:44:05.505094 (+ 261us) client_negotiation.cc:240] Negotiation successful
0808 02:44:05.505598 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4109,"thread_start_us":3920,"threads_started":1}
I20260808 02:44:05.508111 30263 heartbeater.cc:347] Connected to a master server at 127.25.137.254:37801
I20260808 02:44:05.508411 30263 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
I20260808 02:44:05.509207 30263 heartbeater.cc:511] Master 127.25.137.254:37801 requested a full tablet report, sending...
I20260808 02:44:05.511536 29995 ts_manager.cc:194] Registered new tserver with Master: bfa2e39d9a3644698a4137ad08e90b3a (127.25.137.193:43653)
I20260808 02:44:05.513403 29995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:52645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:44:05.534018 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37801
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:05.855050 30270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:05.855311 30270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:05.855441 30270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:05.862613 30270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:05.862711 30270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:05.862859 30270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:44:05.872671 30270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:05.873960 30270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:05.875216 30270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:05.887702 30284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:05.887742 30281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:05.887764 30279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:05.888478 30270 server_base.cc:1034] running on GCE node
I20260808 02:44:05.889149 30270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:05.890060 30270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:05.891273 30270 hybrid_clock.cc:648] HybridClock initialized: now 1786157045891219 us; error 38 us; skew 500 ppm
I20260808 02:44:05.891605 30270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:05 dist-test-slave-k5rr krb5kdc[29935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157045, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:05.894781 30270 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:44:05.896207 30270 webserver.cc:460] Webserver started at http://127.25.137.194:37667/ using document root <none> and password file <none>
I20260808 02:44:05.896945 30270 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:05.897100 30270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:05.897392 30270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:05.900318 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "db4c06bc62b0427ca1306eaa604bb406"
format_stamp: "Formatted at 2026-08-08 02:44:05 on dist-test-slave-k5rr"
server_key: "f4d5e2e2d9f64f21a1e1689722b53ff1"
server_key_iv: "3040f2ff392f7cfdf5e8e6d7fad96da5"
server_key_version: "encryptionkey@0"
I20260808 02:44:05.901144 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "db4c06bc62b0427ca1306eaa604bb406"
format_stamp: "Formatted at 2026-08-08 02:44:05 on dist-test-slave-k5rr"
server_key: "f4d5e2e2d9f64f21a1e1689722b53ff1"
server_key_iv: "3040f2ff392f7cfdf5e8e6d7fad96da5"
server_key_version: "encryptionkey@0"
I20260808 02:44:05.906339 30270 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 02:44:05.909739 30293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:05.911054 30270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:05.911211 30270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "db4c06bc62b0427ca1306eaa604bb406"
format_stamp: "Formatted at 2026-08-08 02:44:05 on dist-test-slave-k5rr"
server_key: "f4d5e2e2d9f64f21a1e1689722b53ff1"
server_key_iv: "3040f2ff392f7cfdf5e8e6d7fad96da5"
server_key_version: "encryptionkey@0"
I20260808 02:44:05.911389 30270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:05.945899 30270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:05.950573 30270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:05.950907 30270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:05.952353 30270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:05.953948 30270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:05.954034 30270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:05.954175 30270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:05.954236 30270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:05.986406 30270 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:45971
I20260808 02:44:05.986449 30468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:45971 every 8 connection(s)
I20260808 02:44:05.987731 30270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:44:05.994279 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 30270
I20260808 02:44:05.994438 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:44:05.994870 26151 external_mini_cluster.cc:1428] Setting key deffc8c8f3dc650b8bcb42bd089f15db
Aug 08 02:44:06 dist-test-slave-k5rr krb5kdc[29935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157045, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:06.009332 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:05.990280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:49655 (local address 127.25.137.254:37801)
0808 02:44:05.990475 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:05.990480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:05.991644 (+ 1164us) server_negotiation.cc:407] Connection header received
0808 02:44:05.993659 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:05.993666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:05.993770 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:05.993870 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:05.995583 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:05.996430 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:05.998270 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:05.998934 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:05.999057 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:06.002683 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:06.002689 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:06.002693 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:06.002736 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:06.005655 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:06.006571 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:06.006574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:06.006577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:06.006700 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:06.007461 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:06.007464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:06.007466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:06.007737 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:06.007849 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:06.008797 (+ 948us) server_negotiation.cc:299] Negotiation successful
0808 02:44:06.009073 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:44:06.010885 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:05.990692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:37801 (local address 127.25.137.194:49655)
0808 02:44:05.991448 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:05.991510 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:05.993373 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:05.994125 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:05.994134 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:05.994738 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:05.995361 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:05.995389 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:05.996611 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:05.996617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:05.998097 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:05.998113 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:05.999146 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:05.999151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:05.999353 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:05.999991 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:06.000019 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:06.002466 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:06.005803 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:06.005814 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:06.005833 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:06.006462 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:06.006805 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:06.006811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:06.006815 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:06.007073 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:06.007873 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:06.007881 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:06.008399 (+ 518us) client_negotiation.cc:769] Sending connection context
0808 02:44:06.008737 (+ 338us) client_negotiation.cc:240] Negotiation successful
0808 02:44:06.009370 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":275,"threads_started":1}
I20260808 02:44:06.012349 30469 heartbeater.cc:347] Connected to a master server at 127.25.137.254:37801
I20260808 02:44:06.012629 30469 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:06.013403 30469 heartbeater.cc:511] Master 127.25.137.254:37801 requested a full tablet report, sending...
I20260808 02:44:06.014969 29995 ts_manager.cc:194] Registered new tserver with Master: db4c06bc62b0427ca1306eaa604bb406 (127.25.137.194:45971)
I20260808 02:44:06.016144 29995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:49655
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:44:06.043903 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37801
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:06.366570 30479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:06.366863 30479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:06.366983 30479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:06.374107 30479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:06.374210 30479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:06.374327 30479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:44:06.383755 30479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:06.385025 30479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:06.386584 30479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:06.399256 30484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:06.399216 30490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:06.399323 30486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:06.400229 30479 server_base.cc:1034] running on GCE node
I20260808 02:44:06.400831 30479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:06.401808 30479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:06.403005 30479 hybrid_clock.cc:648] HybridClock initialized: now 1786157046402943 us; error 51 us; skew 500 ppm
I20260808 02:44:06.403345 30479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:06 dist-test-slave-k5rr krb5kdc[29935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:06.406427 30479 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:44:06.407825 30479 webserver.cc:460] Webserver started at http://127.25.137.195:43613/ using document root <none> and password file <none>
I20260808 02:44:06.408465 30479 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:06.408584 30479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:06.408867 30479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:06.411661 30479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "3dc911d3ac90460ab6924dc43a7cc438"
format_stamp: "Formatted at 2026-08-08 02:44:06 on dist-test-slave-k5rr"
server_key: "6a7b9de091c55e983cf656ac27023777"
server_key_iv: "6fdb688d5dffa7a0baf759e99bf031c6"
server_key_version: "encryptionkey@0"
I20260808 02:44:06.412475 30479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "3dc911d3ac90460ab6924dc43a7cc438"
format_stamp: "Formatted at 2026-08-08 02:44:06 on dist-test-slave-k5rr"
server_key: "6a7b9de091c55e983cf656ac27023777"
server_key_iv: "6fdb688d5dffa7a0baf759e99bf031c6"
server_key_version: "encryptionkey@0"
I20260808 02:44:06.418556 30479 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.002s
I20260808 02:44:06.421996 30499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:06.423244 30479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:44:06.423417 30479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "3dc911d3ac90460ab6924dc43a7cc438"
format_stamp: "Formatted at 2026-08-08 02:44:06 on dist-test-slave-k5rr"
server_key: "6a7b9de091c55e983cf656ac27023777"
server_key_iv: "6fdb688d5dffa7a0baf759e99bf031c6"
server_key_version: "encryptionkey@0"
I20260808 02:44:06.423597 30479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:06.448398 30479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:06.452574 30479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:06.452862 30479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:06.453838 30479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:06.455116 30479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:06.455195 30479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:06.455323 30479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:06.455379 30479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:06.487327 30479 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:39269
I20260808 02:44:06.487375 30667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:39269 every 8 connection(s)
I20260808 02:44:06.488528 30479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:44:06.493698 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 30479
I20260808 02:44:06.493861 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:44:06.494380 26151 external_mini_cluster.cc:1428] Setting key 4051b7cabbef74b216dc7c860d281d5d
Aug 08 02:44:06 dist-test-slave-k5rr krb5kdc[29935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:06.507900 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:06.491008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:48169 (local address 127.25.137.254:37801)
0808 02:44:06.491175 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:06.491179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:06.492322 (+ 1143us) server_negotiation.cc:407] Connection header received
0808 02:44:06.494154 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:06.494160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:06.494285 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:06.494416 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:06.496099 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:06.497000 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:06.498630 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:06.499090 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:06.499201 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:06.502555 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:06.502559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:06.502562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:06.502595 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:06.504990 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:06.505696 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:06.505701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:06.505705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:06.505847 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:06.506255 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:06.506260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:06.506264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:06.506536 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:06.506676 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:06.507334 (+ 658us) server_negotiation.cc:299] Negotiation successful
0808 02:44:06.507553 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:44:06.508864 30673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:06.491428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:37801 (local address 127.25.137.195:48169)
0808 02:44:06.492130 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:06.492175 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:06.493885 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:06.494620 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:06.494628 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:06.495232 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:06.495929 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:06.495948 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:06.497173 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:06.497177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:06.498488 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:06.498504 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:06.499305 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:06.499312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:06.499550 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:06.500151 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:06.500174 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:06.502362 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:06.505146 (+ 2784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:06.505155 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:06.505171 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:06.505583 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:06.505954 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:06.505959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:06.505963 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:06.506148 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:06.506677 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:06.506684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:06.506987 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 02:44:06.507223 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 02:44:06.507706 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":265,"threads_started":1}
I20260808 02:44:06.510114 30669 heartbeater.cc:347] Connected to a master server at 127.25.137.254:37801
I20260808 02:44:06.510407 30669 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:06.511065 30669 heartbeater.cc:511] Master 127.25.137.254:37801 requested a full tablet report, sending...
I20260808 02:44:06.512396 29995 ts_manager.cc:194] Registered new tserver with Master: 3dc911d3ac90460ab6924dc43a7cc438 (127.25.137.195:39269)
I20260808 02:44:06.513218 29995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:48169
I20260808 02:44:06.516371 30263 heartbeater.cc:503] Master 127.25.137.254:37801 was elected leader, sending a full tablet report...
I20260808 02:44:06.521873 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:44:06.557960 30266 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:06.555548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58948 (local address 127.25.137.254:37801)
0808 02:44:06.555717 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:06.555722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:06.556076 (+ 354us) server_negotiation.cc:407] Connection header received
0808 02:44:06.556244 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:06.556247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:06.556339 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:06.556422 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:06.557763 (+ 1341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58948: BlockingRecv error: recv got EOF from 127.0.0.1:58948 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:44:06.561100 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 30078
W20260808 02:44:06.563889 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 30078: Not authorized: Client connection negotiation failed: client connection to 127.25.137.193:43653: server requires authentication, but client does not have Kerberos credentials available
I20260808 02:44:06.564054 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 30078
I20260808 02:44:06.586633 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 30270
W20260808 02:44:06.589249 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 30270: Not authorized: Client connection negotiation failed: client connection to 127.25.137.194:45971: server requires authentication, but client does not have Kerberos credentials available
I20260808 02:44:06.589406 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 30270
I20260808 02:44:06.609604 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 30479
W20260808 02:44:06.612666 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 30479: Not authorized: Client connection negotiation failed: client connection to 127.25.137.195:39269: server requires authentication, but client does not have Kerberos credentials available
I20260808 02:44:06.612833 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 30479
I20260808 02:44:06.633649 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29954
I20260808 02:44:06.634967 30035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:06.738651 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 29954
2026-08-08T02:44:06Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3713 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:44:06 dist-test-slave-k5rr krb5kdc[30697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:44:06 dist-test-slave-k5rr krb5kdc[30697](info): set up 2 sockets
Aug 08 02:44:06 dist-test-slave-k5rr krb5kdc[30697](info): commencing operation
krb5kdc: starting...
W20260808 02:44:07.974646 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:44:08 dist-test-slave-k5rr krb5kdc[30697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:44:08.190836 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:33235
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:33235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:08.515466 30713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:08.515750 30713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:08.515856 30713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:08.522948 30713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:08.523051 30713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:08.523106 30713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:08.523164 30713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:08.523211 30713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:08.533205 30713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:33235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:33235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:08.534399 30713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:08.535665 30713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:08.546872 30724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:08.546940 30725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:08.546970 30728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:08.547919 30713 server_base.cc:1034] running on GCE node
I20260808 02:44:08.548621 30713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:08.549949 30713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:08.551156 30713 hybrid_clock.cc:648] HybridClock initialized: now 1786157048551114 us; error 28 us; skew 500 ppm
I20260808 02:44:08.551505 30713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:08 dist-test-slave-k5rr krb5kdc[30697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157048, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:08.559166 30713 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:44:08.560608 30713 webserver.cc:460] Webserver started at http://127.25.137.254:44873/ using document root <none> and password file <none>
I20260808 02:44:08.561321 30713 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:08.561436 30713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:08.561738 30713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:08.564697 30713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "5bcb2c7690404f9497e461fdf22d0b78"
format_stamp: "Formatted at 2026-08-08 02:44:08 on dist-test-slave-k5rr"
server_key: "10b15e23a4f61176d77bb77b9bf1d96b"
server_key_iv: "87df4b20fe2c2c527eccdbb71a60df05"
server_key_version: "encryptionkey@0"
I20260808 02:44:08.565521 30713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "5bcb2c7690404f9497e461fdf22d0b78"
format_stamp: "Formatted at 2026-08-08 02:44:08 on dist-test-slave-k5rr"
server_key: "10b15e23a4f61176d77bb77b9bf1d96b"
server_key_iv: "87df4b20fe2c2c527eccdbb71a60df05"
server_key_version: "encryptionkey@0"
I20260808 02:44:08.570422 30713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 02:44:08.573746 30737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:08.575157 30713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:08.575304 30713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "5bcb2c7690404f9497e461fdf22d0b78"
format_stamp: "Formatted at 2026-08-08 02:44:08 on dist-test-slave-k5rr"
server_key: "10b15e23a4f61176d77bb77b9bf1d96b"
server_key_iv: "87df4b20fe2c2c527eccdbb71a60df05"
server_key_version: "encryptionkey@0"
I20260808 02:44:08.575481 30713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:08.591966 30713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:08.595767 30713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:08.596035 30713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:08.613389 30713 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:33235
I20260808 02:44:08.613423 30810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:33235 every 8 connection(s)
I20260808 02:44:08.614751 30713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:08.618350 30811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:08.621444 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 30713
I20260808 02:44:08.621667 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:08.622251 26151 external_mini_cluster.cc:1428] Setting key 3a9b74098edc3b5cfd519d51b1dbf341
I20260808 02:44:08.630295 30811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78: Bootstrap starting.
I20260808 02:44:08.633911 30811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:08.635142 30811 log.cc:824] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:44:08 dist-test-slave-k5rr krb5kdc[30697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:08.638116 30811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78: No bootstrap required, opened a new log
I20260808 02:44:08.645265 30816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:08.623961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51624 (local address 127.25.137.254:33235)
0808 02:44:08.624679 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:08.624690 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:08.624728 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:44:08.626392 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:08.626419 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:08.626835 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:08.627149 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:08.628889 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:08.630220 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:08.631936 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:08.632598 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:08.632720 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:08.637281 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:08.637293 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:08.637311 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:08.637354 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:08.641083 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:08.641744 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:08.641753 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:08.641761 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:08.641961 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:08.642415 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:08.642422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:08.642428 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:08.642994 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:08.643315 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:08.643785 (+ 470us) server_negotiation.cc:299] Negotiation successful
0808 02:44:08.644170 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":330,"threads_started":1}
I20260808 02:44:08.646971 30811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcb2c7690404f9497e461fdf22d0b78" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33235 } }
I20260808 02:44:08.647321 30811 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:08.647426 30811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bcb2c7690404f9497e461fdf22d0b78, State: Initialized, Role: FOLLOWER
I20260808 02:44:08.648025 30811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcb2c7690404f9497e461fdf22d0b78" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33235 } }
I20260808 02:44:08.648270 30811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:08.648403 30811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:08.648562 30811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:08.648965 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:44:08.651592 30811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcb2c7690404f9497e461fdf22d0b78" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33235 } }
I20260808 02:44:08.652168 30811 leader_election.cc:302] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bcb2c7690404f9497e461fdf22d0b78; no voters:
I20260808 02:44:08.652802 30811 leader_election.cc:288] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:08.652920 30819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:08.654042 30819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [term 1 LEADER]: Becoming Leader. State: Replica: 5bcb2c7690404f9497e461fdf22d0b78, State: Running, Role: LEADER
I20260808 02:44:08.654517 30819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcb2c7690404f9497e461fdf22d0b78" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33235 } }
I20260808 02:44:08.654884 30811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:08.657040 30820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bcb2c7690404f9497e461fdf22d0b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcb2c7690404f9497e461fdf22d0b78" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33235 } } }
I20260808 02:44:08.657400 30820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:08.658669 30821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bcb2c7690404f9497e461fdf22d0b78. Latest consensus state: current_term: 1 leader_uuid: "5bcb2c7690404f9497e461fdf22d0b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcb2c7690404f9497e461fdf22d0b78" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33235 } } }
I20260808 02:44:08.659046 30821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:08.660422 30826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:08.666060 30826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:08.689718 30826 catalog_manager.cc:1347] Generated new cluster ID: 9247b688d302400da5ec5ff0b3c69850
I20260808 02:44:08.689855 30826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:08.704125 30826 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:08.706107 30826 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:08.709864 30816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:08.699515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.25.137.254:33235)
0808 02:44:08.699700 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:08.699707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:08.702662 (+ 2955us) server_negotiation.cc:407] Connection header received
0808 02:44:08.702861 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:08.702866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:08.702951 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:08.703076 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:08.704409 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:08.705300 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:08.706638 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:08.707083 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:08.707195 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:08.708263 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:08.708268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:08.708271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:08.709045 (+ 774us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:08.709116 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:08.709146 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:08.709244 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:08.709378 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 02:44:08.709509 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":55}
I20260808 02:44:08.714869 30826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bcb2c7690404f9497e461fdf22d0b78: Generated new TSK 0
I20260808 02:44:08.715911 30826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:44:08.725284 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 30713
I20260808 02:44:08.726625 30804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:08.815598 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 30713
2026-08-08T02:44:08Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2075 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:44:08 dist-test-slave-k5rr krb5kdc[30863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:44:08 dist-test-slave-k5rr krb5kdc[30863](info): set up 2 sockets
Aug 08 02:44:08 dist-test-slave-k5rr krb5kdc[30863](info): commencing operation
krb5kdc: starting...
W20260808 02:44:10.013748 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:44:10 dist-test-slave-k5rr krb5kdc[30863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:44:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:44:10.231752 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:40457
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:40457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:10.562049 30890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:10.562363 30890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:10.562518 30890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:10.569919 30890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:10.570036 30890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:10.570114 30890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:10.570199 30890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:10.570286 30890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:10.579905 30890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:40457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:40457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:10.581228 30890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:10.582499 30890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:10.593837 30895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:10.593686 30896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:10.593724 30899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:10.594961 30890 server_base.cc:1034] running on GCE node
I20260808 02:44:10.595705 30890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:10.597112 30890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:10.598340 30890 hybrid_clock.cc:648] HybridClock initialized: now 1786157050598287 us; error 42 us; skew 500 ppm
I20260808 02:44:10.598661 30890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:10 dist-test-slave-k5rr krb5kdc[30863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157050, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:10.601835 30890 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:44:10.603250 30890 webserver.cc:460] Webserver started at http://127.25.137.254:38289/ using document root <none> and password file <none>
I20260808 02:44:10.603874 30890 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:10.603976 30890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:10.604300 30890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:10.607326 30890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "a0adcdc3baed48708b53b5bbe4e2c3c4"
format_stamp: "Formatted at 2026-08-08 02:44:10 on dist-test-slave-k5rr"
server_key: "6ddf23eaba348e5ec6cc0c99c4755913"
server_key_iv: "84f79cb43d42c0a1151f05c736e185f2"
server_key_version: "encryptionkey@0"
I20260808 02:44:10.608033 30890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "a0adcdc3baed48708b53b5bbe4e2c3c4"
format_stamp: "Formatted at 2026-08-08 02:44:10 on dist-test-slave-k5rr"
server_key: "6ddf23eaba348e5ec6cc0c99c4755913"
server_key_iv: "84f79cb43d42c0a1151f05c736e185f2"
server_key_version: "encryptionkey@0"
I20260808 02:44:10.613751 30890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260808 02:44:10.617201 30908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:10.618537 30890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:44:10.618700 30890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "a0adcdc3baed48708b53b5bbe4e2c3c4"
format_stamp: "Formatted at 2026-08-08 02:44:10 on dist-test-slave-k5rr"
server_key: "6ddf23eaba348e5ec6cc0c99c4755913"
server_key_iv: "84f79cb43d42c0a1151f05c736e185f2"
server_key_version: "encryptionkey@0"
I20260808 02:44:10.618901 30890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:10.644394 30890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:10.648454 30890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:10.648736 30890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:10.666065 30890 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:40457
I20260808 02:44:10.666145 30985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:40457 every 8 connection(s)
I20260808 02:44:10.667373 30890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:10.670918 30986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:10.672026 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 30890
I20260808 02:44:10.672226 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:10.672852 26151 external_mini_cluster.cc:1428] Setting key 47f509c0901ea474ece626b3ee5f7339
I20260808 02:44:10.684571 30986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4: Bootstrap starting.
Aug 08 02:44:10 dist-test-slave-k5rr krb5kdc[30863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:10.687474 30986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:10.688745 30986 log.cc:824] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:10.692134 30986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4: No bootstrap required, opened a new log
I20260808 02:44:10.696125 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:10.674917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.25.137.254:40457)
0808 02:44:10.675771 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:10.675785 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:10.675850 (+ 65us) server_negotiation.cc:407] Connection header received
0808 02:44:10.677516 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:10.677543 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:10.678059 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:10.678469 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:10.679683 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:10.681661 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:10.682912 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:10.683907 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:10.684075 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:10.687867 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:10.687879 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:10.687901 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:10.687953 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:10.691743 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:10.692433 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:10.692441 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:10.692449 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:10.692681 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:10.693084 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:10.693089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:10.693092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:10.693766 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:10.694094 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:10.694381 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 02:44:10.694827 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":388,"threads_started":1}
I20260808 02:44:10.701514 30986 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0adcdc3baed48708b53b5bbe4e2c3c4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40457 } }
I20260808 02:44:10.701967 30986 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:10.702070 30986 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0adcdc3baed48708b53b5bbe4e2c3c4, State: Initialized, Role: FOLLOWER
I20260808 02:44:10.702702 30986 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0adcdc3baed48708b53b5bbe4e2c3c4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40457 } }
I20260808 02:44:10.702932 30986 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:10.703073 30986 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:10.703224 30986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:10.705454 30986 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0adcdc3baed48708b53b5bbe4e2c3c4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40457 } }
I20260808 02:44:10.705925 30986 leader_election.cc:302] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0adcdc3baed48708b53b5bbe4e2c3c4; no voters:
I20260808 02:44:10.706573 30986 leader_election.cc:288] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:10.706678 30992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:10.707371 30992 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [term 1 LEADER]: Becoming Leader. State: Replica: a0adcdc3baed48708b53b5bbe4e2c3c4, State: Running, Role: LEADER
I20260808 02:44:10.707887 30992 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0adcdc3baed48708b53b5bbe4e2c3c4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40457 } }
I20260808 02:44:10.708582 30986 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:10.710539 30995 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0adcdc3baed48708b53b5bbe4e2c3c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0adcdc3baed48708b53b5bbe4e2c3c4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40457 } } }
I20260808 02:44:10.710923 30995 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:10.710934 30996 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0adcdc3baed48708b53b5bbe4e2c3c4. Latest consensus state: current_term: 1 leader_uuid: "a0adcdc3baed48708b53b5bbe4e2c3c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0adcdc3baed48708b53b5bbe4e2c3c4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40457 } } }
I20260808 02:44:10.712004 30996 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:10.713799 31004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:10.718734 31004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:10.727607 31004 catalog_manager.cc:1347] Generated new cluster ID: 7b38312a171d4fd28e0c84349aad9f85
I20260808 02:44:10.727725 31004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:10.750427 31004 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:10.752345 31004 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:10.759231 31004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0adcdc3baed48708b53b5bbe4e2c3c4: Generated new TSK 0
I20260808 02:44:10.760265 31004 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:44:10.815269 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:40457
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:11.143913 31023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:11.144184 31023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:11.144279 31023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:11.151432 31023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:11.151562 31023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:11.151690 31023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:44:11.161505 31023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:40457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:11.162803 31023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:11.164053 31023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:11.176998 31030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:11.177222 31031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:11.176963 31033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:11.177835 31023 server_base.cc:1034] running on GCE node
I20260808 02:44:11.178480 31023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:11.179376 31023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:11.180572 31023 hybrid_clock.cc:648] HybridClock initialized: now 1786157051180523 us; error 40 us; skew 500 ppm
I20260808 02:44:11.180863 31023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:11 dist-test-slave-k5rr krb5kdc[30863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157051, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:11.184090 31023 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:44:11.185617 31023 webserver.cc:460] Webserver started at http://127.25.137.193:43805/ using document root <none> and password file <none>
I20260808 02:44:11.186277 31023 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:11.186396 31023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:11.186685 31023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:11.189579 31023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "de3821a45b124f6aa8b90da4a92efad1"
format_stamp: "Formatted at 2026-08-08 02:44:11 on dist-test-slave-k5rr"
server_key: "22a7a7c946e7a7fb77af5936a299caad"
server_key_iv: "907ccee20d0708f8e03082cee272d46c"
server_key_version: "encryptionkey@0"
I20260808 02:44:11.190383 31023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "de3821a45b124f6aa8b90da4a92efad1"
format_stamp: "Formatted at 2026-08-08 02:44:11 on dist-test-slave-k5rr"
server_key: "22a7a7c946e7a7fb77af5936a299caad"
server_key_iv: "907ccee20d0708f8e03082cee272d46c"
server_key_version: "encryptionkey@0"
I20260808 02:44:11.195592 31023 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:44:11.199008 31042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:11.200366 31023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:44:11.200501 31023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "de3821a45b124f6aa8b90da4a92efad1"
format_stamp: "Formatted at 2026-08-08 02:44:11 on dist-test-slave-k5rr"
server_key: "22a7a7c946e7a7fb77af5936a299caad"
server_key_iv: "907ccee20d0708f8e03082cee272d46c"
server_key_version: "encryptionkey@0"
I20260808 02:44:11.200659 31023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:11.219192 31023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:11.223798 31023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:11.224093 31023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:11.225126 31023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:11.226523 31023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:11.226601 31023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:11.226715 31023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:11.226783 31023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:11.261372 31023 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:44717
I20260808 02:44:11.261425 31215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:44717 every 8 connection(s)
I20260808 02:44:11.262717 31023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:44:11.267694 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 31023
I20260808 02:44:11.267918 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:44:11.268635 26151 external_mini_cluster.cc:1428] Setting key 088d8de36ccd8dd15d85731c88b3e087
Aug 08 02:44:11 dist-test-slave-k5rr krb5kdc[30863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157051, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:11.284121 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:11.265424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:59535 (local address 127.25.137.254:40457)
0808 02:44:11.265932 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:11.265937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:11.266947 (+ 1010us) server_negotiation.cc:407] Connection header received
0808 02:44:11.268861 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:11.268867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:11.268947 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:11.269091 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:11.270819 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:11.271544 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:11.273099 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:11.273502 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:11.273619 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:11.277561 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:11.277566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:11.277569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:11.277603 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:11.280640 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:11.281505 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:11.281510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:11.281513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:11.281657 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:11.282176 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:11.282182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:11.282185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:11.282560 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:11.282704 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:11.283517 (+ 813us) server_negotiation.cc:299] Negotiation successful
0808 02:44:11.283744 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":236,"threads_started":1}
I20260808 02:44:11.284914 31220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:11.265975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:40457 (local address 127.25.137.193:59535)
0808 02:44:11.266713 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:11.266760 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:11.268552 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:11.269328 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:11.269337 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:11.269955 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:11.270650 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:11.270669 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:11.271704 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:11.271708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:11.272937 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:11.272949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:11.273732 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:11.273740 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:11.273970 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:11.274602 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:11.274631 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:11.277292 (+ 2661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:11.280805 (+ 3513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:11.280812 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:11.280827 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:11.281366 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:11.281820 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:11.281824 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:11.281826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:11.282036 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:11.282719 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:11.282725 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:11.283116 (+ 391us) client_negotiation.cc:769] Sending connection context
0808 02:44:11.283358 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 02:44:11.283769 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":299,"threads_started":1}
I20260808 02:44:11.286553 31216 heartbeater.cc:347] Connected to a master server at 127.25.137.254:40457
I20260808 02:44:11.286923 31216 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:11.287850 31216 heartbeater.cc:511] Master 127.25.137.254:40457 requested a full tablet report, sending...
I20260808 02:44:11.290181 30932 ts_manager.cc:194] Registered new tserver with Master: de3821a45b124f6aa8b90da4a92efad1 (127.25.137.193:44717)
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
I20260808 02:44:11.291654 30932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:59535
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:44:11.319762 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:40457
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:11.648298 31224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:11.648588 31224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:11.648729 31224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:11.656337 31224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:11.656458 31224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:11.656625 31224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:44:11.666523 31224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:40457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:11.667766 31224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:11.669044 31224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:11.681864 31232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:11.681962 31235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:11.681938 31233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:11.683027 31224 server_base.cc:1034] running on GCE node
I20260808 02:44:11.683610 31224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:11.684551 31224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:11.685745 31224 hybrid_clock.cc:648] HybridClock initialized: now 1786157051685691 us; error 49 us; skew 500 ppm
I20260808 02:44:11.686084 31224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:11 dist-test-slave-k5rr krb5kdc[30863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157051, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:11.689213 31224 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:44:11.690610 31224 webserver.cc:460] Webserver started at http://127.25.137.194:35157/ using document root <none> and password file <none>
I20260808 02:44:11.691272 31224 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:11.691393 31224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:11.691694 31224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:11.694700 31224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "47e5b3ef8bc54c21a5e6654fb2c8da71"
format_stamp: "Formatted at 2026-08-08 02:44:11 on dist-test-slave-k5rr"
server_key: "ae565fa39e81a0ed14cc96fe58641d4d"
server_key_iv: "764d0a2fe4132092d71a981f99513873"
server_key_version: "encryptionkey@0"
I20260808 02:44:11.695484 31224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "47e5b3ef8bc54c21a5e6654fb2c8da71"
format_stamp: "Formatted at 2026-08-08 02:44:11 on dist-test-slave-k5rr"
server_key: "ae565fa39e81a0ed14cc96fe58641d4d"
server_key_iv: "764d0a2fe4132092d71a981f99513873"
server_key_version: "encryptionkey@0"
I20260808 02:44:11.700696 31224 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 02:44:11.704226 31243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:11.705698 31224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:44:11.705891 31224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "47e5b3ef8bc54c21a5e6654fb2c8da71"
format_stamp: "Formatted at 2026-08-08 02:44:11 on dist-test-slave-k5rr"
server_key: "ae565fa39e81a0ed14cc96fe58641d4d"
server_key_iv: "764d0a2fe4132092d71a981f99513873"
server_key_version: "encryptionkey@0"
I20260808 02:44:11.706075 31224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:11.732057 31224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:11.736140 31224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:11.736426 31224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:11.737413 31224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:11.738754 31224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:11.738847 31224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:11.738965 31224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:11.739017 31224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:11.769644 31224 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:46123
I20260808 02:44:11.769701 31415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:46123 every 8 connection(s)
I20260808 02:44:11.770905 31224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:44:11.771317 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 31224
I20260808 02:44:11.771512 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:44:11.772085 26151 external_mini_cluster.cc:1428] Setting key 847c7589b4ab8ac73ee6bcd4724e3767
Aug 08 02:44:11 dist-test-slave-k5rr krb5kdc[30863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157051, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:11.791378 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:11.773490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:53051 (local address 127.25.137.254:40457)
0808 02:44:11.773656 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:11.773660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:11.775114 (+ 1454us) server_negotiation.cc:407] Connection header received
0808 02:44:11.777079 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:11.777085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:11.777167 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:11.777270 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:11.778773 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:11.779699 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:11.781247 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:11.781786 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:11.781911 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:11.785769 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:11.785773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:11.785777 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:11.785813 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:11.788257 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:11.788988 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:11.788992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:11.788994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:11.789152 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:11.789574 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:11.789579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:11.789583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:11.789874 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:11.790021 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:11.790786 (+ 765us) server_negotiation.cc:299] Negotiation successful
0808 02:44:11.791020 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:44:11.792390 31422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:11.774149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:40457 (local address 127.25.137.194:53051)
0808 02:44:11.774935 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:11.774990 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:11.776719 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:11.777481 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:11.777490 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:11.778008 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:11.778591 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:11.778609 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:11.779848 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:11.779852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:11.781107 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:11.781120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:11.781979 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:11.781984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:11.782183 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:11.782934 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:11.782964 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:11.785547 (+ 2583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:11.788419 (+ 2872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:11.788427 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:11.788445 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:11.788873 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:11.789260 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:11.789265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:11.789269 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:11.789461 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:11.790008 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:11.790016 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:11.790427 (+ 411us) client_negotiation.cc:769] Sending connection context
0808 02:44:11.790672 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 02:44:11.791223 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":319,"threads_started":1}
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
I20260808 02:44:11.793889 31416 heartbeater.cc:347] Connected to a master server at 127.25.137.254:40457
I20260808 02:44:11.794164 31416 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:11.795136 31416 heartbeater.cc:511] Master 127.25.137.254:40457 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
I20260808 02:44:11.796816 30932 ts_manager.cc:194] Registered new tserver with Master: 47e5b3ef8bc54c21a5e6654fb2c8da71 (127.25.137.194:46123)
I20260808 02:44:11.797709 30932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:53051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:44:11.821758 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:40457
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:46223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:12.149849 31425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:12.150133 31425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:12.150240 31425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:12.157455 31425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:12.157557 31425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:12.157685 31425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:44:12.167454 31425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:46223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:40457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:12.168778 31425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:12.170069 31425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:12.182690 31434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:12.182637 31433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:12.183112 31437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:12.183601 31425 server_base.cc:1034] running on GCE node
I20260808 02:44:12.184185 31425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:12.185113 31425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:12.186309 31425 hybrid_clock.cc:648] HybridClock initialized: now 1786157052186244 us; error 58 us; skew 500 ppm
I20260808 02:44:12.186601 31425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:12 dist-test-slave-k5rr krb5kdc[30863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157052, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:12.189731 31425 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:44:12.191139 31425 webserver.cc:460] Webserver started at http://127.25.137.195:39445/ using document root <none> and password file <none>
I20260808 02:44:12.191808 31425 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:12.191921 31425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:12.192212 31425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:12.195128 31425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "53f6fd1cee8f431eb391480b2242182f"
format_stamp: "Formatted at 2026-08-08 02:44:12 on dist-test-slave-k5rr"
server_key: "09e3fcdf8249b5630dee7c0f14a40a36"
server_key_iv: "af48c9f78ffa4baf5f961a6de17bb596"
server_key_version: "encryptionkey@0"
I20260808 02:44:12.195883 31425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "53f6fd1cee8f431eb391480b2242182f"
format_stamp: "Formatted at 2026-08-08 02:44:12 on dist-test-slave-k5rr"
server_key: "09e3fcdf8249b5630dee7c0f14a40a36"
server_key_iv: "af48c9f78ffa4baf5f961a6de17bb596"
server_key_version: "encryptionkey@0"
I20260808 02:44:12.200886 31425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:44:12.204486 31448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:12.205907 31425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:44:12.206044 31425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "53f6fd1cee8f431eb391480b2242182f"
format_stamp: "Formatted at 2026-08-08 02:44:12 on dist-test-slave-k5rr"
server_key: "09e3fcdf8249b5630dee7c0f14a40a36"
server_key_iv: "af48c9f78ffa4baf5f961a6de17bb596"
server_key_version: "encryptionkey@0"
I20260808 02:44:12.206203 31425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:12.226572 31425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:12.230749 31425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:12.231010 31425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:12.231940 31425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:12.233357 31425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:12.233435 31425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:12.233522 31425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:12.233566 31425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:12.264986 31425 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:41511
I20260808 02:44:12.265064 31611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:41511 every 8 connection(s)
I20260808 02:44:12.266238 31425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:44:12.273638 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 31425
I20260808 02:44:12.273866 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:44:12.274329 26151 external_mini_cluster.cc:1428] Setting key 23c9d6f5a8639f4927c456253e8e201c
Aug 08 02:44:12 dist-test-slave-k5rr krb5kdc[30863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157052, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:12.286434 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:12.268724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:55407 (local address 127.25.137.254:40457)
0808 02:44:12.268926 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:12.268931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:12.270106 (+ 1175us) server_negotiation.cc:407] Connection header received
0808 02:44:12.271982 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:12.271988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:12.272095 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:12.272186 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:12.273985 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.274911 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.276922 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.277608 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.277728 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:12.281158 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:12.281163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:12.281167 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:12.281200 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:12.283700 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:12.284400 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:12.284404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:12.284406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:12.284533 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:12.284888 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:12.284891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:12.284893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:12.285205 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:12.285317 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:12.286004 (+ 687us) server_negotiation.cc:299] Negotiation successful
0808 02:44:12.286174 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:44:12.287328 31615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:12.269213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:40457 (local address 127.25.137.195:55407)
0808 02:44:12.269930 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:12.269970 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:12.271712 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:12.272449 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:12.272457 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:12.272976 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:12.273790 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:12.273817 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.275094 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.275099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:12.276763 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:12.276776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.277815 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.277823 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:12.278020 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:12.278632 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:12.278658 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:12.280933 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:12.283839 (+ 2906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:12.283845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:12.283858 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:12.284266 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:12.284613 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:12.284616 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:12.284618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:12.284791 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:12.285344 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:12.285351 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:12.285658 (+ 307us) client_negotiation.cc:769] Sending connection context
0808 02:44:12.285889 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 02:44:12.286265 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":272,"threads_started":1}
I20260808 02:44:12.288795 31612 heartbeater.cc:347] Connected to a master server at 127.25.137.254:40457
I20260808 02:44:12.289134 31612 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:12.289992 31612 heartbeater.cc:511] Master 127.25.137.254:40457 requested a full tablet report, sending...
I20260808 02:44:12.291329 30932 ts_manager.cc:194] Registered new tserver with Master: 53f6fd1cee8f431eb391480b2242182f (127.25.137.195:41511)
I20260808 02:44:12.292131 30932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:55407
I20260808 02:44:12.294883 31216 heartbeater.cc:503] Master 127.25.137.254:40457 was elected leader, sending a full tablet report...
I20260808 02:44:12.301934 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:44:12 dist-test-slave-k5rr krb5kdc[30863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157052, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 02:44:12 dist-test-slave-k5rr krb5kdc[30863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157052, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:12.354676 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:12.340967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48394 (local address 127.25.137.254:40457)
0808 02:44:12.341236 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:12.341242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:12.341616 (+ 374us) server_negotiation.cc:407] Connection header received
0808 02:44:12.341781 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:12.341785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:12.341861 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:12.341939 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:12.343292 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.344022 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.345299 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.345744 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.345873 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:12.349434 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:12.349439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:12.349443 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:12.349483 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:12.352322 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:12.352917 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:12.352921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:12.352923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:12.353134 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:12.353458 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:12.353461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:12.353463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:12.353783 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:12.353966 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:12.354197 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 02:44:12.354377 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
W20260808 02:44:12.355221 30932 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48394
I20260808 02:44:12.356830 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 31023
Aug 08 02:44:12 dist-test-slave-k5rr krb5kdc[30863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157052, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:44:12.371628 31626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:12.357778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40368 (local address 127.25.137.193:44717)
0808 02:44:12.358364 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:12.358371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:12.358395 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:44:12.358450 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:12.358456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:12.358650 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:12.358772 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:12.359925 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.360941 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.362059 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.362638 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.362734 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:12.366453 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:12.366466 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:12.366481 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:12.366524 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:12.369233 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:12.369805 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:12.369812 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:12.369814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:12.369946 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:12.370235 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:12.370238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:12.370240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:12.370592 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:12.370811 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:12.371059 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 02:44:12.371255 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":268,"threads_started":1}
W20260808 02:44:12.372871 31211 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40368
W20260808 02:44:12.374022 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 31023: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:44:12.374195 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 31023
I20260808 02:44:12.395658 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 31224
Aug 08 02:44:12 dist-test-slave-k5rr krb5kdc[30863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157052, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:44:12.412084 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:12.396504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36264 (local address 127.25.137.194:46123)
0808 02:44:12.397068 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:12.397076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:12.397102 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:44:12.397161 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:12.397167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:12.397371 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:12.397491 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:12.398793 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.399795 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.401156 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.401713 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.401818 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:12.405779 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:12.405794 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:12.405809 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:12.405856 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:12.408867 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:12.409621 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:12.409636 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:12.409640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:12.409843 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:12.410278 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:12.410282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:12.410284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:12.410653 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:12.411002 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:12.411410 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 02:44:12.411714 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":242,"threads_started":1}
W20260808 02:44:12.413437 31410 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36264
W20260808 02:44:12.414487 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 31224: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:44:12.414646 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 31224
I20260808 02:44:12.439055 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 31425
Aug 08 02:44:12 dist-test-slave-k5rr krb5kdc[30863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157052, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:44:12.455947 31628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:12.440108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.25.137.195:41511)
0808 02:44:12.440737 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:12.440744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:12.440775 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:44:12.440844 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:12.440850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:12.441196 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:12.441430 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:12.442806 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.443839 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.445079 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:12.445648 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:12.445765 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:12.449975 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:12.449986 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:12.450000 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:12.450038 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:12.452961 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:12.453672 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:12.453679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:12.453681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:12.453822 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:12.454220 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:12.454224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:12.454226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:12.454607 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:12.454909 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:12.455288 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 02:44:12.455566 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":330,"threads_started":1}
W20260808 02:44:12.457320 31604 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53484
W20260808 02:44:12.458439 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 31425: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:44:12.458587 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 31425
I20260808 02:44:12.484882 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 30890
I20260808 02:44:12.486294 30981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:12.589778 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 30890
2026-08-08T02:44:12Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3777 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:44:12 dist-test-slave-k5rr krb5kdc[31633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:44:12 dist-test-slave-k5rr krb5kdc[31633](info): set up 2 sockets
Aug 08 02:44:12 dist-test-slave-k5rr krb5kdc[31633](info): commencing operation
krb5kdc: starting...
W20260808 02:44:13.805555 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.005s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:44:13 dist-test-slave-k5rr krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:44:14.022979 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:34679
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:34679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:14.344862 31658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:14.345186 31658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:14.345342 31658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:14.352407 31658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:14.352516 31658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:14.352604 31658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:14.352680 31658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:14.352743 31658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:14.362649 31658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:34679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:34679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:14.363942 31658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:14.365259 31658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:14.376984 31663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:14.377056 31668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:14.377081 31664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:14.377919 31658 server_base.cc:1034] running on GCE node
I20260808 02:44:14.378649 31658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:14.379967 31658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:14.381176 31658 hybrid_clock.cc:648] HybridClock initialized: now 1786157054381113 us; error 47 us; skew 500 ppm
I20260808 02:44:14.381543 31658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:14 dist-test-slave-k5rr krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157054, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:14.387581 31658 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:44:14.389026 31658 webserver.cc:460] Webserver started at http://127.25.137.254:41895/ using document root <none> and password file <none>
I20260808 02:44:14.389693 31658 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:14.389812 31658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:14.390116 31658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:14.393126 31658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "7395dab80531431497164f18dfde1d74"
format_stamp: "Formatted at 2026-08-08 02:44:14 on dist-test-slave-k5rr"
server_key: "a40dde8426c95635c50b657d902a3a99"
server_key_iv: "e02f4ec1c04367c2ddeea1485cfe516f"
server_key_version: "encryptionkey@0"
I20260808 02:44:14.393953 31658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "7395dab80531431497164f18dfde1d74"
format_stamp: "Formatted at 2026-08-08 02:44:14 on dist-test-slave-k5rr"
server_key: "a40dde8426c95635c50b657d902a3a99"
server_key_iv: "e02f4ec1c04367c2ddeea1485cfe516f"
server_key_version: "encryptionkey@0"
I20260808 02:44:14.398851 31658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 02:44:14.402120 31683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:14.403434 31658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:14.403590 31658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "7395dab80531431497164f18dfde1d74"
format_stamp: "Formatted at 2026-08-08 02:44:14 on dist-test-slave-k5rr"
server_key: "a40dde8426c95635c50b657d902a3a99"
server_key_iv: "e02f4ec1c04367c2ddeea1485cfe516f"
server_key_version: "encryptionkey@0"
I20260808 02:44:14.403764 31658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:14.424244 31658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:14.428011 31658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:14.428270 31658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:14.445788 31658 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:34679
I20260808 02:44:14.445812 31759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:34679 every 8 connection(s)
I20260808 02:44:14.447118 31658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:14.450665 31763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:14.453691 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 31658
I20260808 02:44:14.453883 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:14.454514 26151 external_mini_cluster.cc:1428] Setting key 8e27f4ae0ce37c1fef214f57ba0010b3
I20260808 02:44:14.463135 31763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74: Bootstrap starting.
I20260808 02:44:14.466683 31763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:14.467935 31763 log.cc:824] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:44:14 dist-test-slave-k5rr krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:14.471457 31763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74: No bootstrap required, opened a new log
I20260808 02:44:14.478761 31769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:14.456479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48578 (local address 127.25.137.254:34679)
0808 02:44:14.457275 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:14.457286 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:14.457334 (+ 48us) server_negotiation.cc:407] Connection header received
0808 02:44:14.458961 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:14.458983 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:14.459372 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:14.459672 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:14.461807 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:14.463084 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:14.464761 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:14.465787 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:14.465972 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:14.470787 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:14.470795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:14.470839 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:14.470882 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:14.474490 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:14.475315 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:14.475321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:14.475326 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:14.475514 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:14.475924 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:14.475927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:14.475929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:14.476486 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:14.476711 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:14.477248 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 02:44:14.477630 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":347,"threads_started":1}
I20260808 02:44:14.480232 31763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7395dab80531431497164f18dfde1d74" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34679 } }
I20260808 02:44:14.480618 31763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:14.480720 31763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7395dab80531431497164f18dfde1d74, State: Initialized, Role: FOLLOWER
I20260808 02:44:14.481344 31763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7395dab80531431497164f18dfde1d74" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34679 } }
I20260808 02:44:14.481585 31763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:14.481720 31763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:14.481873 31763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:14.483915 31763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7395dab80531431497164f18dfde1d74" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34679 } }
I20260808 02:44:14.484380 31763 leader_election.cc:302] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7395dab80531431497164f18dfde1d74; no voters:
I20260808 02:44:14.484920 31763 leader_election.cc:288] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:14.485085 31772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:14.485730 31772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [term 1 LEADER]: Becoming Leader. State: Replica: 7395dab80531431497164f18dfde1d74, State: Running, Role: LEADER
I20260808 02:44:14.486294 31772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7395dab80531431497164f18dfde1d74" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34679 } }
I20260808 02:44:14.487012 31763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:14.488979 31776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7395dab80531431497164f18dfde1d74. Latest consensus state: current_term: 1 leader_uuid: "7395dab80531431497164f18dfde1d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7395dab80531431497164f18dfde1d74" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34679 } } }
I20260808 02:44:14.488953 31773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7395dab80531431497164f18dfde1d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7395dab80531431497164f18dfde1d74" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34679 } } }
I20260808 02:44:14.489408 31776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:14.489573 31773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:14.491382 31781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:14.495844 31781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:14.504769 31781 catalog_manager.cc:1347] Generated new cluster ID: 6ef0a8eebb834f698e12961f4573b929
I20260808 02:44:14.504904 31781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:14.553087 31781 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:14.554970 31781 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:14.565912 31781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7395dab80531431497164f18dfde1d74: Generated new TSK 0
I20260808 02:44:14.566870 31781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:44:14.634770 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34679
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:14.960716 31803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:14.961028 31803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:14.961169 31803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:14.968357 31803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:14.968492 31803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:14.968647 31803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:44:14.978585 31803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:14.979851 31803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:14.981305 31803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:14.994446 31810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:14.994470 31811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:14.995152 31803 server_base.cc:1034] running on GCE node
W20260808 02:44:14.995261 31813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:14.995965 31803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:14.996884 31803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:14.998095 31803 hybrid_clock.cc:648] HybridClock initialized: now 1786157054998050 us; error 56 us; skew 500 ppm
I20260808 02:44:14.998399 31803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:14 dist-test-slave-k5rr krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157054, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:15.001525 31803 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:44:15.002935 31803 webserver.cc:460] Webserver started at http://127.25.137.193:45451/ using document root <none> and password file <none>
I20260808 02:44:15.003587 31803 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:15.003701 31803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:15.004009 31803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:15.007078 31803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "7c81578276dd46188d432d671536fd6c"
format_stamp: "Formatted at 2026-08-08 02:44:14 on dist-test-slave-k5rr"
server_key: "9d395365d0577819c37bc3cf16f9c10e"
server_key_iv: "f8bed8457d3425f48d58dead61a45a3c"
server_key_version: "encryptionkey@0"
I20260808 02:44:15.007889 31803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "7c81578276dd46188d432d671536fd6c"
format_stamp: "Formatted at 2026-08-08 02:44:14 on dist-test-slave-k5rr"
server_key: "9d395365d0577819c37bc3cf16f9c10e"
server_key_iv: "f8bed8457d3425f48d58dead61a45a3c"
server_key_version: "encryptionkey@0"
I20260808 02:44:15.013296 31803 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:44:15.016717 31822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:15.018067 31803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:44:15.018224 31803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "7c81578276dd46188d432d671536fd6c"
format_stamp: "Formatted at 2026-08-08 02:44:14 on dist-test-slave-k5rr"
server_key: "9d395365d0577819c37bc3cf16f9c10e"
server_key_iv: "f8bed8457d3425f48d58dead61a45a3c"
server_key_version: "encryptionkey@0"
I20260808 02:44:15.018409 31803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:15.031302 31803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:15.035807 31803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:15.036109 31803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:15.037160 31803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:15.038625 31803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:15.038718 31803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:15.038872 31803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:15.038935 31803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:15.071403 31803 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:36127
I20260808 02:44:15.071442 31998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:36127 every 8 connection(s)
I20260808 02:44:15.072788 31803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:44:15.077351 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 31803
I20260808 02:44:15.077524 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:44:15.078147 26151 external_mini_cluster.cc:1428] Setting key b713794ffa7d5233e951e9e53cd3eb24
Aug 08 02:44:15 dist-test-slave-k5rr krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157054, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:15.093056 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:15.075507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:35435 (local address 127.25.137.254:34679)
0808 02:44:15.075946 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:15.075950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:15.076904 (+ 954us) server_negotiation.cc:407] Connection header received
0808 02:44:15.078820 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:15.078825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:15.078895 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:15.078987 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:15.080578 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:15.081613 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:15.083163 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:15.083705 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:15.083831 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:15.087206 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:15.087212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:15.087216 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:15.087254 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:15.089905 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:15.090757 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:15.090760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:15.090763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:15.090879 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:15.091314 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:15.091317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:15.091319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:15.091594 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:15.091702 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:15.092584 (+ 882us) server_negotiation.cc:299] Negotiation successful
0808 02:44:15.092748 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":217,"threads_started":1}
I20260808 02:44:15.094401 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:15.076027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34679 (local address 127.25.137.193:35435)
0808 02:44:15.076705 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:15.076780 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:15.078541 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:15.079259 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:15.079268 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:15.079846 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:15.080407 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:15.080425 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:15.081790 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:15.081796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:15.083017 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:15.083031 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:15.083922 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:15.083928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:15.084116 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:15.084742 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:15.084764 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:15.087006 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:15.090044 (+ 3038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:15.090053 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:15.090069 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:15.090658 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:15.090981 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:15.090986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:15.090989 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:15.091215 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:15.091728 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:15.091737 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:15.092169 (+ 432us) client_negotiation.cc:769] Sending connection context
0808 02:44:15.092484 (+ 315us) client_negotiation.cc:240] Negotiation successful
0808 02:44:15.092996 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":258,"threads_started":1}
I20260808 02:44:15.096068 31999 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34679
I20260808 02:44:15.096416 31999 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:15.097155 31999 heartbeater.cc:511] Master 127.25.137.254:34679 requested a full tablet report, sending...
I20260808 02:44:15.099223 31711 ts_manager.cc:194] Registered new tserver with Master: 7c81578276dd46188d432d671536fd6c (127.25.137.193:36127)
I20260808 02:44:15.100641 31711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:35435
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:44:15.130365 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34679
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:15.466934 32006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:15.467247 32006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:15.467397 32006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:15.475018 32006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:15.475147 32006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:15.475334 32006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:44:15.487955 32006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:15.489327 32006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:15.490687 32006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:15.503897 32013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:15.503959 32014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:15.503993 32016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:15.505125 32006 server_base.cc:1034] running on GCE node
I20260808 02:44:15.505813 32006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:15.506896 32006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:15.508144 32006 hybrid_clock.cc:648] HybridClock initialized: now 1786157055508086 us; error 55 us; skew 500 ppm
I20260808 02:44:15.508490 32006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:15 dist-test-slave-k5rr krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157055, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:15.511649 32006 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:44:15.513119 32006 webserver.cc:460] Webserver started at http://127.25.137.194:42709/ using document root <none> and password file <none>
I20260808 02:44:15.513762 32006 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:15.513862 32006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:15.514111 32006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:15.516920 32006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "0cff3100073d445f984b57ae8caf9a14"
format_stamp: "Formatted at 2026-08-08 02:44:15 on dist-test-slave-k5rr"
server_key: "356158c3448cdcfa2f0a3202722a3e04"
server_key_iv: "a6219d18ee6a5ea2a37e29ff0fd744ec"
server_key_version: "encryptionkey@0"
I20260808 02:44:15.517701 32006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "0cff3100073d445f984b57ae8caf9a14"
format_stamp: "Formatted at 2026-08-08 02:44:15 on dist-test-slave-k5rr"
server_key: "356158c3448cdcfa2f0a3202722a3e04"
server_key_iv: "a6219d18ee6a5ea2a37e29ff0fd744ec"
server_key_version: "encryptionkey@0"
I20260808 02:44:15.522642 32006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:44:15.526068 32028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:15.527316 32006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:44:15.527472 32006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "0cff3100073d445f984b57ae8caf9a14"
format_stamp: "Formatted at 2026-08-08 02:44:15 on dist-test-slave-k5rr"
server_key: "356158c3448cdcfa2f0a3202722a3e04"
server_key_iv: "a6219d18ee6a5ea2a37e29ff0fd744ec"
server_key_version: "encryptionkey@0"
I20260808 02:44:15.527649 32006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:15.564043 32006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:15.568216 32006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:15.568493 32006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:15.569502 32006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:15.570860 32006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:15.570938 32006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:15.571067 32006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:15.571123 32006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:15.602005 32006 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:38039
I20260808 02:44:15.602073 32209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:38039 every 8 connection(s)
I20260808 02:44:15.603257 32006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:44:15.611982 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 32006
I20260808 02:44:15.612175 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:44:15.612715 26151 external_mini_cluster.cc:1428] Setting key 1f4b72e96ea6f6d0052018285800142e
Aug 08 02:44:15 dist-test-slave-k5rr krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157055, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:15.624907 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:15.605666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:54531 (local address 127.25.137.254:34679)
0808 02:44:15.606131 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:15.606135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:15.607077 (+ 942us) server_negotiation.cc:407] Connection header received
0808 02:44:15.609110 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:15.609115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:15.609189 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:15.609283 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:15.610788 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:15.611531 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:15.613823 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:15.614549 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:15.614700 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:15.618622 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:15.618629 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:15.618633 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:15.618672 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:15.621608 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:15.622377 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:15.622383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:15.622386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:15.622541 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:15.623101 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:15.623105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:15.623109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:15.623454 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:15.623588 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:15.624312 (+ 724us) server_negotiation.cc:299] Negotiation successful
0808 02:44:15.624539 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":235,"threads_started":1}
I20260808 02:44:15.625864 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:15.606229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34679 (local address 127.25.137.194:54531)
0808 02:44:15.606905 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:15.606943 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:15.608826 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:15.609508 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:15.609516 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:15.610066 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:15.610621 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:15.610639 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:15.611712 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:15.611717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:15.613594 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:15.613609 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:15.614789 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:15.614798 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:15.615039 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:15.615744 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:15.615785 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:15.618386 (+ 2601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:15.621759 (+ 3373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:15.621766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:15.621778 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:15.622253 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:15.622672 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:15.622675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:15.622678 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:15.622973 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:15.623567 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:15.623573 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:15.623929 (+ 356us) client_negotiation.cc:769] Sending connection context
0808 02:44:15.624167 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 02:44:15.624674 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":255,"threads_started":1}
I20260808 02:44:15.627352 32210 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34679
I20260808 02:44:15.627626 32210 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:15.628309 32210 heartbeater.cc:511] Master 127.25.137.254:34679 requested a full tablet report, sending...
I20260808 02:44:15.629732 31711 ts_manager.cc:194] Registered new tserver with Master: 0cff3100073d445f984b57ae8caf9a14 (127.25.137.194:38039)
I20260808 02:44:15.630851 31711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:54531
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:44:15.662264 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34679
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:15.983372 32218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:15.983634 32218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:15.983728 32218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:15.991168 32218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:15.991268 32218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:15.991395 32218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:44:16.000897 32218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:16.002177 32218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:16.003481 32218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:16.016350 32226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:16.016367 32230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:16.016454 32227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:16.017822 32218 server_base.cc:1034] running on GCE node
I20260808 02:44:16.018399 32218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:16.019310 32218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:16.020516 32218 hybrid_clock.cc:648] HybridClock initialized: now 1786157056020456 us; error 38 us; skew 500 ppm
I20260808 02:44:16.020833 32218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:16 dist-test-slave-k5rr krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157056, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:16.023988 32218 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:44:16.025542 32218 webserver.cc:460] Webserver started at http://127.25.137.195:45851/ using document root <none> and password file <none>
I20260808 02:44:16.026206 32218 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:16.026324 32218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:16.026620 32218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:16.029665 32218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "f3b3f8de06c647528bbc20131b95dda3"
format_stamp: "Formatted at 2026-08-08 02:44:16 on dist-test-slave-k5rr"
server_key: "b791916dd7433cee82c0c8c437d4a42e"
server_key_iv: "8692e8063a96d75ff66d6dfd412c609b"
server_key_version: "encryptionkey@0"
I20260808 02:44:16.030450 32218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "f3b3f8de06c647528bbc20131b95dda3"
format_stamp: "Formatted at 2026-08-08 02:44:16 on dist-test-slave-k5rr"
server_key: "b791916dd7433cee82c0c8c437d4a42e"
server_key_iv: "8692e8063a96d75ff66d6dfd412c609b"
server_key_version: "encryptionkey@0"
I20260808 02:44:16.035522 32218 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:44:16.038808 32237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:16.040086 32218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:44:16.040247 32218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "f3b3f8de06c647528bbc20131b95dda3"
format_stamp: "Formatted at 2026-08-08 02:44:16 on dist-test-slave-k5rr"
server_key: "b791916dd7433cee82c0c8c437d4a42e"
server_key_iv: "8692e8063a96d75ff66d6dfd412c609b"
server_key_version: "encryptionkey@0"
I20260808 02:44:16.040458 32218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:16.056818 32218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:16.061066 32218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:16.061355 32218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:16.062244 32218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:16.063580 32218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:16.063652 32218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:16.063748 32218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:16.063802 32218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:16.093864 32218 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:35877
I20260808 02:44:16.093961 32400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:35877 every 8 connection(s)
I20260808 02:44:16.095112 32218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:44:16.102216 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 32218
I20260808 02:44:16.102392 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:44:16.102909 26151 external_mini_cluster.cc:1428] Setting key 9dbbbb47fd6916c4a8eae2ee1dfe8e04
I20260808 02:44:16.103720 31999 heartbeater.cc:503] Master 127.25.137.254:34679 was elected leader, sending a full tablet report...
Aug 08 02:44:16 dist-test-slave-k5rr krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157056, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:16.115598 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:16.097662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:59213 (local address 127.25.137.254:34679)
0808 02:44:16.097851 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:16.097855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:16.099171 (+ 1316us) server_negotiation.cc:407] Connection header received
0808 02:44:16.101051 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:16.101058 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:16.101152 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:16.101239 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:16.102985 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.104051 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.106243 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.106631 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.106745 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:16.110267 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:16.110272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:16.110275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:16.110311 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:16.112676 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.113441 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.113444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.113447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.113578 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.113987 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.113990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.113992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.114280 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:16.114409 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:16.115174 (+ 765us) server_negotiation.cc:299] Negotiation successful
0808 02:44:16.115360 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:44:16.116750 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:16.098125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34679 (local address 127.25.137.195:59213)
0808 02:44:16.098983 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:16.099038 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:16.100746 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:16.101524 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:16.101534 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:16.102136 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:16.102787 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:16.102815 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.104261 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.104267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:16.106106 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:16.106123 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.106854 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.106862 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:16.107086 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:16.107747 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:16.107772 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:16.110064 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:16.112801 (+ 2737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:16.112848 (+ 47us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:16.112860 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:16.113332 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:16.113663 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:16.113667 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:16.113669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:16.113900 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:16.114434 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:16.114441 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:16.114845 (+ 404us) client_negotiation.cc:769] Sending connection context
0808 02:44:16.115116 (+ 271us) client_negotiation.cc:240] Negotiation successful
0808 02:44:16.115529 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":305,"threads_started":1}
I20260808 02:44:16.118103 32404 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34679
I20260808 02:44:16.118469 32404 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:16.119311 32404 heartbeater.cc:511] Master 127.25.137.254:34679 requested a full tablet report, sending...
I20260808 02:44:16.120723 31709 ts_manager.cc:194] Registered new tserver with Master: f3b3f8de06c647528bbc20131b95dda3 (127.25.137.195:35877)
I20260808 02:44:16.121775 31709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:59213
I20260808 02:44:16.131045 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:44:16 dist-test-slave-k5rr krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:44:16 dist-test-slave-k5rr krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:44:16.184010 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:16.169512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.25.137.193:36127)
0808 02:44:16.170137 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:16.170144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:16.170171 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:44:16.170235 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:16.170241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:16.170456 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:16.170585 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:16.171991 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.173056 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.174208 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.174745 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.174835 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:16.178569 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:16.178582 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:16.178596 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:16.178635 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:16.181356 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.181971 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.181978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.181981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.182107 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.182474 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.182477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.182479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.182809 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:16.183109 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:16.183387 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 02:44:16.183649 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":296,"threads_started":1}
Aug 08 02:44:16 dist-test-slave-k5rr krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:16.200302 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:16.188012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.25.137.254:34679)
0808 02:44:16.188286 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:16.188292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:16.188417 (+ 125us) server_negotiation.cc:407] Connection header received
0808 02:44:16.188579 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:16.188583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:16.188647 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:16.188725 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:16.189885 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.190581 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.191786 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.192203 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.192292 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:16.195719 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:16.195723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:16.195726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:16.195759 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:16.198199 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.198729 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.198732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.198734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.198854 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.199130 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.199132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.199134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.199421 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:16.199605 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:16.199817 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 02:44:16.199992 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 02:44:16.200872 31709 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48584
I20260808 02:44:16.201890 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 31803
I20260808 02:44:16.213604 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:16.202884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46424 (local address 127.25.137.193:36127)
0808 02:44:16.203067 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:16.203073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:16.203296 (+ 223us) server_negotiation.cc:407] Connection header received
0808 02:44:16.203413 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:16.203417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:16.203479 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:16.203552 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:16.204650 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.205393 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.206550 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.206993 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.207079 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:16.209170 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:16.209175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:16.209178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:16.209208 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:16.211436 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.211989 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.211992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.211995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.212101 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.212395 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.212398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.212400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.212627 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:16.212805 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:16.213136 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 02:44:16.213301 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:44:16.214228 31989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:16.296708 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 31803
I20260808 02:44:16.317294 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 32006
Aug 08 02:44:16 dist-test-slave-k5rr krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:44:16.333112 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:16.318169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46504 (local address 127.25.137.194:38039)
0808 02:44:16.318761 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:16.318769 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:16.318795 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:44:16.318867 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:16.318874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:16.319077 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:16.319211 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:16.320453 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.321543 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.322841 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.323400 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.323500 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:16.327626 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:16.327638 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:16.327652 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:16.327695 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:16.330440 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.331048 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.331055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.331058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.331182 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.331556 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.331559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.331562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.331937 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:16.332167 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:16.332407 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 02:44:16.332604 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":293,"threads_started":1}
I20260808 02:44:16.334419 32200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:16.418488 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 32006
I20260808 02:44:16.443110 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 32218
Aug 08 02:44:16 dist-test-slave-k5rr krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:44:16.460531 32426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:16.444397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38758 (local address 127.25.137.195:35877)
0808 02:44:16.445127 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:16.445135 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:16.445164 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:44:16.445230 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:16.445236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:16.445464 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:16.445640 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:16.446934 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.448437 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.450460 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:16.451032 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:16.451146 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:16.455393 (+ 4247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:16.455405 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:16.455419 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:16.455454 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:16.458064 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.458660 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.458666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.458669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.458784 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:16.459136 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:16.459140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:16.459142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:16.459545 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:16.459828 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:16.460041 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 02:44:16.460245 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":339,"threads_started":1}
I20260808 02:44:16.461875 32393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:16.545388 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 32218
I20260808 02:44:16.565893 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 31658
I20260808 02:44:16.567149 31754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:16.672240 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 31658
2026-08-08T02:44:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4084 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:44:16 dist-test-slave-k5rr krb5kdc[32436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:44:16 dist-test-slave-k5rr krb5kdc[32436](info): set up 2 sockets
Aug 08 02:44:16 dist-test-slave-k5rr krb5kdc[32436](info): commencing operation
krb5kdc: starting...
W20260808 02:44:17.898777 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:44:18 dist-test-slave-k5rr krb5kdc[32436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:44:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:44:18.122051 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:33107
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:33107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:18.448738 32452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:18.449102 32452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:18.449241 32452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:18.457322 32452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:18.457446 32452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:18.457526 32452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:18.457607 32452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:18.457672 32452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:18.467651 32452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:33107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:33107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:18.468958 32452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:18.470312 32452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:18.481281 32459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:18.481309 32460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:18.481353 32462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:18.482699 32452 server_base.cc:1034] running on GCE node
I20260808 02:44:18.483501 32452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:18.485069 32452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:18.486368 32452 hybrid_clock.cc:648] HybridClock initialized: now 1786157058486308 us; error 48 us; skew 500 ppm
I20260808 02:44:18.486724 32452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:18 dist-test-slave-k5rr krb5kdc[32436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157058, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:18.489850 32452 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:44:18.490967 32452 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:18.491071 32452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:18.491411 32452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:18.494369 32452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "08d1db3e1ccf49ab80e255a63dd90d71"
format_stamp: "Formatted at 2026-08-08 02:44:18 on dist-test-slave-k5rr"
server_key: "93f05a1b3b7814e68badad2e89a6833d"
server_key_iv: "c769e1562862cdf104d1bad4c0d028a2"
server_key_version: "encryptionkey@0"
I20260808 02:44:18.495149 32452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "08d1db3e1ccf49ab80e255a63dd90d71"
format_stamp: "Formatted at 2026-08-08 02:44:18 on dist-test-slave-k5rr"
server_key: "93f05a1b3b7814e68badad2e89a6833d"
server_key_iv: "c769e1562862cdf104d1bad4c0d028a2"
server_key_version: "encryptionkey@0"
I20260808 02:44:18.500006 32452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260808 02:44:18.503414 32471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:18.504951 32452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:44:18.505151 32452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "08d1db3e1ccf49ab80e255a63dd90d71"
format_stamp: "Formatted at 2026-08-08 02:44:18 on dist-test-slave-k5rr"
server_key: "93f05a1b3b7814e68badad2e89a6833d"
server_key_iv: "c769e1562862cdf104d1bad4c0d028a2"
server_key_version: "encryptionkey@0"
I20260808 02:44:18.505376 32452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:18.524508 32452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:18.529026 32452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:18.529301 32452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:18.547569 32452 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:33107
I20260808 02:44:18.547582 32550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:33107 every 8 connection(s)
I20260808 02:44:18.548467 32452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:18.552191 32555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:18.553959 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 32452
I20260808 02:44:18.554153 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:18.554859 26151 external_mini_cluster.cc:1428] Setting key b9da703111523ecca1878704a38ca917
I20260808 02:44:18.565037 32555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71: Bootstrap starting.
I20260808 02:44:18.568475 32555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:18.569482 32555 log.cc:824] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:44:18 dist-test-slave-k5rr krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:18.572788 32555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71: No bootstrap required, opened a new log
I20260808 02:44:18.578490 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:18.556967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46228 (local address 127.25.137.254:33107)
0808 02:44:18.557872 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:18.557887 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:18.557938 (+ 51us) server_negotiation.cc:407] Connection header received
0808 02:44:18.560185 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:18.560214 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:18.560698 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:18.561160 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:18.562578 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:18.564469 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:18.565866 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:18.566889 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:18.567061 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:18.571111 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:18.571121 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:18.571140 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:18.571189 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:18.574513 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:18.575197 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:18.575203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:18.575209 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:18.575389 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:18.575755 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:18.575758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:18.575760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:18.576309 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:18.576612 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:18.577120 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 02:44:18.577503 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":365,"threads_started":1}
I20260808 02:44:18.580824 32555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d1db3e1ccf49ab80e255a63dd90d71" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33107 } }
I20260808 02:44:18.581291 32555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:18.581403 32555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08d1db3e1ccf49ab80e255a63dd90d71, State: Initialized, Role: FOLLOWER
I20260808 02:44:18.581924 32555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d1db3e1ccf49ab80e255a63dd90d71" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33107 } }
I20260808 02:44:18.582137 32555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:18.582259 32555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:18.582405 32555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:18.584705 32555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d1db3e1ccf49ab80e255a63dd90d71" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33107 } }
I20260808 02:44:18.585206 32555 leader_election.cc:302] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08d1db3e1ccf49ab80e255a63dd90d71; no voters:
I20260808 02:44:18.585736 32555 leader_election.cc:288] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:18.585855 32569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:18.586491 32569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [term 1 LEADER]: Becoming Leader. State: Replica: 08d1db3e1ccf49ab80e255a63dd90d71, State: Running, Role: LEADER
I20260808 02:44:18.586988 32569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d1db3e1ccf49ab80e255a63dd90d71" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33107 } }
I20260808 02:44:18.587713 32555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:18.589520 32571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08d1db3e1ccf49ab80e255a63dd90d71. Latest consensus state: current_term: 1 leader_uuid: "08d1db3e1ccf49ab80e255a63dd90d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d1db3e1ccf49ab80e255a63dd90d71" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33107 } } }
I20260808 02:44:18.589603 32570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08d1db3e1ccf49ab80e255a63dd90d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08d1db3e1ccf49ab80e255a63dd90d71" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 33107 } } }
I20260808 02:44:18.589957 32570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:18.589953 32571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:18.590840 32575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:18.598912 32575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:18.607095 32575 catalog_manager.cc:1347] Generated new cluster ID: 001eb471acd84c809399073f6c3b300d
I20260808 02:44:18.607203 32575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:18.651391 32575 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:18.652715 32575 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:18.658413 32575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71: Generated new TSK 0
I20260808 02:44:18.659245 32575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:44:18.729928 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:33107
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:19.079088 32602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:19.079375 32602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:19.079519 32602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:19.086737 32602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:19.086865 32602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:19.087016 32602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:44:19.096961 32602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:33107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:19.098260 32602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:19.099534 32602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:19.111709 32608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:19.111711 32611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:19.111757 32609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:19.112984 32602 server_base.cc:1034] running on GCE node
I20260808 02:44:19.113606 32602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:19.114527 32602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:19.115732 32602 hybrid_clock.cc:648] HybridClock initialized: now 1786157059115679 us; error 44 us; skew 500 ppm
I20260808 02:44:19.116043 32602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:19 dist-test-slave-k5rr krb5kdc[32436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157059, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:19.119176 32602 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:44:19.120297 32602 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:19.120389 32602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:19.120653 32602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:19.123570 32602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "80e517527f3145c4a933aee1e5c6c830"
format_stamp: "Formatted at 2026-08-08 02:44:19 on dist-test-slave-k5rr"
server_key: "6973706c7b69e66321187281ae67d8a0"
server_key_iv: "2af3c26619a7e9820c21e02f83a0c5a4"
server_key_version: "encryptionkey@0"
I20260808 02:44:19.124364 32602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "80e517527f3145c4a933aee1e5c6c830"
format_stamp: "Formatted at 2026-08-08 02:44:19 on dist-test-slave-k5rr"
server_key: "6973706c7b69e66321187281ae67d8a0"
server_key_iv: "2af3c26619a7e9820c21e02f83a0c5a4"
server_key_version: "encryptionkey@0"
I20260808 02:44:19.129367 32602 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 02:44:19.132663 32620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:19.134024 32602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:44:19.134160 32602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "80e517527f3145c4a933aee1e5c6c830"
format_stamp: "Formatted at 2026-08-08 02:44:19 on dist-test-slave-k5rr"
server_key: "6973706c7b69e66321187281ae67d8a0"
server_key_iv: "2af3c26619a7e9820c21e02f83a0c5a4"
server_key_version: "encryptionkey@0"
I20260808 02:44:19.134356 32602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:19.151664 32602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:19.155721 32602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:19.155975 32602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:19.156893 32602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:19.158250 32602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:19.158341 32602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:19.158447 32602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:19.158529 32602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:19.188534 32602 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:46403
I20260808 02:44:19.188573 328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:46403 every 8 connection(s)
I20260808 02:44:19.189409 32602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:44:19.195525 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 32602
I20260808 02:44:19.195712 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:44:19.196323 26151 external_mini_cluster.cc:1428] Setting key 43595a465143cc490b3258ab844df28a
Aug 08 02:44:19 dist-test-slave-k5rr krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157059, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:19.208511 333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:19.191807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:59677 (local address 127.25.137.254:33107)
0808 02:44:19.192238 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:19.192244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:19.193159 (+ 915us) server_negotiation.cc:407] Connection header received
0808 02:44:19.194969 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:19.194975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:19.195076 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:19.195192 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:19.196795 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:19.197622 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:19.199161 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:19.199558 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:19.199668 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:19.203144 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:19.203149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:19.203153 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:19.203186 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:19.205603 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:19.206292 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:19.206296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:19.206298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:19.206418 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:19.206809 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:19.206814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:19.206817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:19.207096 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:19.207227 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:19.207927 (+ 700us) server_negotiation.cc:299] Negotiation successful
0808 02:44:19.208145 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":222,"threads_started":1}
I20260808 02:44:19.209528 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:19.192284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:33107 (local address 127.25.137.193:59677)
0808 02:44:19.192996 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:19.193061 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:19.194701 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:19.195413 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:19.195421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:19.196004 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:19.196616 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:19.196637 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:19.197784 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:19.197788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:19.199004 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:19.199018 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:19.199774 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:19.199782 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:19.200009 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:19.200653 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:19.200674 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:19.202925 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:19.205746 (+ 2821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:19.205755 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:19.205772 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:19.206178 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:19.206519 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:19.206524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:19.206528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:19.206703 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:19.207227 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:19.207235 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:19.207558 (+ 323us) client_negotiation.cc:769] Sending connection context
0808 02:44:19.207800 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 02:44:19.208352 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":275,"threads_started":1}
I20260808 02:44:19.211024 329 heartbeater.cc:347] Connected to a master server at 127.25.137.254:33107
I20260808 02:44:19.211302 329 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:19.211956 329 heartbeater.cc:511] Master 127.25.137.254:33107 requested a full tablet report, sending...
I20260808 02:44:19.213984 32493 ts_manager.cc:194] Registered new tserver with Master: 80e517527f3145c4a933aee1e5c6c830 (127.25.137.193:46403)
I20260808 02:44:19.215365 32493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:59677
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:44:19.246037 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:33107
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:19.571722 337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:19.571997 337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:19.572095 337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:19.579403 337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:19.579512 337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:19.579628 337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:44:19.589304 337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:33107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:19.590531 337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:19.591847 337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:19.604207 343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:19.604241 349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:19.604291 344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:19.605365 337 server_base.cc:1034] running on GCE node
I20260808 02:44:19.605928 337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:19.606798 337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:19.607987 337 hybrid_clock.cc:648] HybridClock initialized: now 1786157059607934 us; error 48 us; skew 500 ppm
I20260808 02:44:19.608289 337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:19 dist-test-slave-k5rr krb5kdc[32436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157059, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:19.614192 337 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:44:19.615326 337 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:19.615432 337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:19.615725 337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:19.618665 337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "330a40e751c44513a3d7110d8724671b"
format_stamp: "Formatted at 2026-08-08 02:44:19 on dist-test-slave-k5rr"
server_key: "060bb3f624ed98cb17c15d9ba6f855dd"
server_key_iv: "1cc26229c520a4e0ca61cb5dab5ab71d"
server_key_version: "encryptionkey@0"
I20260808 02:44:19.619462 337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "330a40e751c44513a3d7110d8724671b"
format_stamp: "Formatted at 2026-08-08 02:44:19 on dist-test-slave-k5rr"
server_key: "060bb3f624ed98cb17c15d9ba6f855dd"
server_key_iv: "1cc26229c520a4e0ca61cb5dab5ab71d"
server_key_version: "encryptionkey@0"
I20260808 02:44:19.624181 337 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:44:19.627435 360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:19.628679 337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:44:19.628841 337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "330a40e751c44513a3d7110d8724671b"
format_stamp: "Formatted at 2026-08-08 02:44:19 on dist-test-slave-k5rr"
server_key: "060bb3f624ed98cb17c15d9ba6f855dd"
server_key_iv: "1cc26229c520a4e0ca61cb5dab5ab71d"
server_key_version: "encryptionkey@0"
I20260808 02:44:19.629109 337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:19.643916 337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:19.647905 337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:19.648211 337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:19.649166 337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:19.650527 337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:19.650629 337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:19.650717 337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:19.650787 337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:19.683820 337 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:34677
I20260808 02:44:19.683907 513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:34677 every 8 connection(s)
I20260808 02:44:19.684814 337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:44:19.689045 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 337
I20260808 02:44:19.689241 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:44:19.689800 26151 external_mini_cluster.cc:1428] Setting key 2c2199dc0ec7b2e13deb77b18cd27ff7
Aug 08 02:44:19 dist-test-slave-k5rr krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157059, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:19.706035 333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:19.687396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:46475 (local address 127.25.137.254:33107)
0808 02:44:19.687575 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:19.687580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:19.688785 (+ 1205us) server_negotiation.cc:407] Connection header received
0808 02:44:19.690943 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:19.690949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:19.691034 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:19.691161 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:19.692879 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:19.693630 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:19.695185 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:19.695634 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:19.695752 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:19.699309 (+ 3557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:19.699315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:19.699318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:19.699355 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:19.702399 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:19.703364 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:19.703369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:19.703371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:19.703512 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:19.704072 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:19.704075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:19.704077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:19.704384 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:19.704511 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:19.705595 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0808 02:44:19.705786 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:44:19.707624 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:19.687849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:33107 (local address 127.25.137.194:46475)
0808 02:44:19.688621 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:19.688670 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:19.690610 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:19.691342 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:19.691351 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:19.692067 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:19.692684 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:19.692711 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:19.693794 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:19.693805 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:19.695041 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:19.695054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:19.695888 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:19.695897 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:19.696099 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:19.696775 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:19.696843 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:19.699064 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:19.702567 (+ 3503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:19.702578 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:19.702595 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:19.703215 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:19.703627 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:19.703632 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:19.703635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:19.703944 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:19.704542 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:19.704550 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:19.705134 (+ 584us) client_negotiation.cc:769] Sending connection context
0808 02:44:19.705498 (+ 364us) client_negotiation.cc:240] Negotiation successful
0808 02:44:19.706051 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":307,"threads_started":1}
I20260808 02:44:19.709333 514 heartbeater.cc:347] Connected to a master server at 127.25.137.254:33107
I20260808 02:44:19.709662 514 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:19.710568 514 heartbeater.cc:511] Master 127.25.137.254:33107 requested a full tablet report, sending...
I20260808 02:44:19.712230 32493 ts_manager.cc:194] Registered new tserver with Master: 330a40e751c44513a3d7110d8724671b (127.25.137.194:34677)
I20260808 02:44:19.713095 32493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:46475
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:44:19.744007 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:33107
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:20.064657 524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:20.064945 524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:20.065114 524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:20.072183 524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:20.072305 524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:20.072459 524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:44:20.082015 524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:33107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:20.083307 524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:20.084609 524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:20.097188 532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:20.097213 533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:20.097252 537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:20.098356 524 server_base.cc:1034] running on GCE node
I20260808 02:44:20.098953 524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:20.099870 524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:20.101099 524 hybrid_clock.cc:648] HybridClock initialized: now 1786157060101045 us; error 43 us; skew 500 ppm
I20260808 02:44:20.101409 524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:20 dist-test-slave-k5rr krb5kdc[32436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157060, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:20.104512 524 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:44:20.105700 524 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:20.105849 524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:20.106150 524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:20.109136 524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "61683f87c863475ab5cca82ec0179317"
format_stamp: "Formatted at 2026-08-08 02:44:20 on dist-test-slave-k5rr"
server_key: "dafcf50a1836340ab9c23e377b8709cb"
server_key_iv: "ff335a127a5b0f4279bb36f2038e8e1d"
server_key_version: "encryptionkey@0"
I20260808 02:44:20.109946 524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "61683f87c863475ab5cca82ec0179317"
format_stamp: "Formatted at 2026-08-08 02:44:20 on dist-test-slave-k5rr"
server_key: "dafcf50a1836340ab9c23e377b8709cb"
server_key_iv: "ff335a127a5b0f4279bb36f2038e8e1d"
server_key_version: "encryptionkey@0"
I20260808 02:44:20.115028 524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:44:20.118458 547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:20.119763 524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:20.119901 524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "61683f87c863475ab5cca82ec0179317"
format_stamp: "Formatted at 2026-08-08 02:44:20 on dist-test-slave-k5rr"
server_key: "dafcf50a1836340ab9c23e377b8709cb"
server_key_iv: "ff335a127a5b0f4279bb36f2038e8e1d"
server_key_version: "encryptionkey@0"
I20260808 02:44:20.120092 524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:20.143329 524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:20.147496 524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:20.147727 524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:20.148703 524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:20.150163 524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:20.150252 524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:20.150336 524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:20.150388 524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:20.180837 524 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:46371
I20260808 02:44:20.180914 713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:46371 every 8 connection(s)
I20260808 02:44:20.181694 524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:44:20.184863 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 524
I20260808 02:44:20.185072 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:44:20.185673 26151 external_mini_cluster.cc:1428] Setting key f0d6df20321c1e2093e8141d51ad23e1
Aug 08 02:44:20 dist-test-slave-k5rr krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157060, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:20.202147 333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.183956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:35815 (local address 127.25.137.254:33107)
0808 02:44:20.184119 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.184123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.185484 (+ 1361us) server_negotiation.cc:407] Connection header received
0808 02:44:20.187660 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.187665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.187774 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.187886 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:20.189776 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.190895 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.192597 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.193290 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.193412 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.196597 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:20.196602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:20.196606 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:20.196639 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:20.198979 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:20.199802 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:20.199807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:20.199809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:20.199989 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:20.200462 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:20.200465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:20.200467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:20.200810 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:20.200976 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.201738 (+ 762us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.201916 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:44:20.203029 719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.184403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:33107 (local address 127.25.137.195:35815)
0808 02:44:20.185301 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.185353 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:20.187396 (+ 2043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:20.188207 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:20.188218 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:20.188877 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:20.189589 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.189607 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.191040 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.191046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.192429 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.192442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.193482 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.193487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.193672 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.194290 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:20.194314 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:20.196404 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:20.199130 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:20.199139 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:20.199153 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:20.199688 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:20.200108 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:20.200113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:20.200116 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:20.200358 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:20.200943 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:20.200949 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:20.201349 (+ 400us) client_negotiation.cc:769] Sending connection context
0808 02:44:20.201626 (+ 277us) client_negotiation.cc:240] Negotiation successful
0808 02:44:20.202009 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":350,"threads_started":1}
I20260808 02:44:20.204325 715 heartbeater.cc:347] Connected to a master server at 127.25.137.254:33107
I20260808 02:44:20.204651 715 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:20.205366 715 heartbeater.cc:511] Master 127.25.137.254:33107 requested a full tablet report, sending...
I20260808 02:44:20.206832 32493 ts_manager.cc:194] Registered new tserver with Master: 61683f87c863475ab5cca82ec0179317 (127.25.137.195:46371)
I20260808 02:44:20.207680 32493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:35815
I20260808 02:44:20.213603 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:44:20.218616 329 heartbeater.cc:503] Master 127.25.137.254:33107 was elected leader, sending a full tablet report...
I20260808 02:44:20.230631 333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.218262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46244 (local address 127.25.137.254:33107)
0808 02:44:20.218435 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.218440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.219029 (+ 589us) server_negotiation.cc:407] Connection header received
0808 02:44:20.219193 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.219198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.219295 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.219382 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:20.221048 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.221747 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.222965 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.223441 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.223527 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.225798 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:20.225803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:20.225806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:20.225839 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:20.228133 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:20.228718 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:20.228722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:20.228724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:20.228866 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:20.229289 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:20.229294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:20.229296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:20.229645 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:20.229863 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.230099 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.230287 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:44:20.234813 32493 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:44:20.241699 32493 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:44:20.267747 744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.254256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:34677 (local address 127.0.0.1:39896)
0808 02:44:20.257991 (+ 3735us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.258010 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:20.259205 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:20.260415 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:20.260425 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:20.260459 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:20.260906 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.260927 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.262770 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.262776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.264912 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.264931 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.266813 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.266820 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.267030 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.267085 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 02:44:20.267175 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 02:44:20.267321 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3606,"mutex_wait_us":29,"thread_start_us":5499,"threads_started":1}
I20260808 02:44:20.268435 741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.255313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:46371 (local address 127.0.0.1:57874)
0808 02:44:20.260021 (+ 4708us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.260038 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:20.260233 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:20.261031 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:20.261035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:20.261060 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:20.261607 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.261621 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.263705 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.263708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.265623 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.265638 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.267601 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.267609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.267792 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.267840 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 02:44:20.267921 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 02:44:20.268046 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4588,"mutex_wait_us":19,"thread_start_us":2408,"threads_started":1}
I20260808 02:44:20.268762 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.253713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46403 (local address 127.0.0.1:56062)
0808 02:44:20.254398 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.254452 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:20.254621 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:20.261066 (+ 6445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:20.261070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:20.261116 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:20.261575 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.261589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.263684 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.263690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.265815 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.265831 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.267867 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.267874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.268049 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.268086 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:44:20.268147 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 02:44:20.268252 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":77,"spinlock_wait_cycles":2816,"thread_start_us":759,"threads_started":1}
I20260808 02:44:20.269143 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.254522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39896 (local address 127.25.137.194:34677)
0808 02:44:20.259709 (+ 5187us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.259717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.259746 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:44:20.259869 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.259877 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.260136 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.260397 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:20.261167 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.262586 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.265115 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.266578 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.268384 (+ 1806us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.268554 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.268642 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.268812 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5044,"thread_start_us":4911,"threads_started":1}
I20260808 02:44:20.269187 745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.255907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57874 (local address 127.25.137.195:46371)
0808 02:44:20.257566 (+ 1659us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.257576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.260133 (+ 2557us) server_negotiation.cc:407] Connection header received
0808 02:44:20.260426 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.260434 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.260637 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.260764 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:20.261842 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.263513 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.265814 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.267367 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.267592 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.267819 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.268675 (+ 856us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.268841 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1464,"thread_start_us":1297,"threads_started":1}
I20260808 02:44:20.269876 739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.254053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56062 (local address 127.25.137.193:46403)
0808 02:44:20.260458 (+ 6405us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.260467 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.260498 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:44:20.260598 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.260607 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.260839 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.261112 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:20.261812 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.263534 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.266052 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.267590 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.269179 (+ 1589us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.269375 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.269457 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.269603 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6225,"thread_start_us":6038,"threads_started":1}
I20260808 02:44:20.273126 428 tablet_service.cc:1467] Processing CreateTablet for tablet 662acda0d0ea4ca0a0b7ff68155f2d18 (DEFAULT_TABLE table=test-table [id=142e58da73fb42dab5d849c30ef5b2b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:20.273211 623 tablet_service.cc:1467] Processing CreateTablet for tablet 662acda0d0ea4ca0a0b7ff68155f2d18 (DEFAULT_TABLE table=test-table [id=142e58da73fb42dab5d849c30ef5b2b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:20.273259 32697 tablet_service.cc:1467] Processing CreateTablet for tablet 662acda0d0ea4ca0a0b7ff68155f2d18 (DEFAULT_TABLE table=test-table [id=142e58da73fb42dab5d849c30ef5b2b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:20.274459 428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 662acda0d0ea4ca0a0b7ff68155f2d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:20.274613 623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 662acda0d0ea4ca0a0b7ff68155f2d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:20.274611 32697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 662acda0d0ea4ca0a0b7ff68155f2d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:20.287091 754 tablet_bootstrap.cc:492] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: Bootstrap starting.
I20260808 02:44:20.289265 755 tablet_bootstrap.cc:492] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: Bootstrap starting.
I20260808 02:44:20.291903 754 tablet_bootstrap.cc:654] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:20.292848 756 tablet_bootstrap.cc:492] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: Bootstrap starting.
I20260808 02:44:20.293675 754 log.cc:824] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:20.294617 755 tablet_bootstrap.cc:654] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:20.295579 755 log.cc:824] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:20.296654 756 tablet_bootstrap.cc:654] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:20.297937 756 log.cc:824] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:20.298408 754 tablet_bootstrap.cc:492] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: No bootstrap required, opened a new log
I20260808 02:44:20.298686 754 ts_tablet_manager.cc:1397] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260808 02:44:20.298807 755 tablet_bootstrap.cc:492] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: No bootstrap required, opened a new log
I20260808 02:44:20.299062 755 ts_tablet_manager.cc:1397] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 02:44:20.301151 756 tablet_bootstrap.cc:492] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: No bootstrap required, opened a new log
I20260808 02:44:20.301420 756 ts_tablet_manager.cc:1397] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260808 02:44:20.306159 754 raft_consensus.cc:348] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 } } peers { permanent_uuid: "61683f87c863475ab5cca82ec0179317" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46371 } } peers { permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 } }
I20260808 02:44:20.306501 754 raft_consensus.cc:374] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:20.306603 754 raft_consensus.cc:729] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 330a40e751c44513a3d7110d8724671b, State: Initialized, Role: FOLLOWER
I20260808 02:44:20.307145 754 consensus_queue.cc:260] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 } } peers { permanent_uuid: "61683f87c863475ab5cca82ec0179317" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46371 } } peers { permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 } }
I20260808 02:44:20.308158 514 heartbeater.cc:503] Master 127.25.137.254:33107 was elected leader, sending a full tablet report...
I20260808 02:44:20.308947 754 ts_tablet_manager.cc:1428] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: Time spent starting tablet: real 0.010s user 0.008s sys 0.004s
I20260808 02:44:20.309983 755 raft_consensus.cc:348] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 } } peers { permanent_uuid: "61683f87c863475ab5cca82ec0179317" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46371 } } peers { permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 } }
I20260808 02:44:20.310405 755 raft_consensus.cc:374] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:20.310501 755 raft_consensus.cc:729] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e517527f3145c4a933aee1e5c6c830, State: Initialized, Role: FOLLOWER
I20260808 02:44:20.311061 755 consensus_queue.cc:260] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 } } peers { permanent_uuid: "61683f87c863475ab5cca82ec0179317" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46371 } } peers { permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 } }
I20260808 02:44:20.311982 756 raft_consensus.cc:348] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 } } peers { permanent_uuid: "61683f87c863475ab5cca82ec0179317" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46371 } } peers { permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 } }
I20260808 02:44:20.312407 756 raft_consensus.cc:374] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:20.312520 756 raft_consensus.cc:729] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61683f87c863475ab5cca82ec0179317, State: Initialized, Role: FOLLOWER
I20260808 02:44:20.312896 755 ts_tablet_manager.cc:1428] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260808 02:44:20.313195 756 consensus_queue.cc:260] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 } } peers { permanent_uuid: "61683f87c863475ab5cca82ec0179317" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46371 } } peers { permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 } }
I20260808 02:44:20.314153 715 heartbeater.cc:503] Master 127.25.137.254:33107 was elected leader, sending a full tablet report...
I20260808 02:44:20.315532 756 ts_tablet_manager.cc:1428] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260808 02:44:20.429795 764 raft_consensus.cc:482] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:20.430121 764 raft_consensus.cc:504] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 } } peers { permanent_uuid: "61683f87c863475ab5cca82ec0179317" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46371 } } peers { permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 } }
I20260808 02:44:20.431831 764 leader_election.cc:288] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80e517527f3145c4a933aee1e5c6c830 (127.25.137.193:46403), 330a40e751c44513a3d7110d8724671b (127.25.137.194:34677)
W20260808 02:44:20.433189 717 tablet.cc:2369] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:44:20.436779 515 tablet.cc:2369] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:44:20.436772 763 raft_consensus.cc:482] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:20.437175 763 raft_consensus.cc:504] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 } } peers { permanent_uuid: "61683f87c863475ab5cca82ec0179317" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46371 } } peers { permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 } }
I20260808 02:44:20.438814 719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.432126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46403 (local address 127.25.137.195:33687)
0808 02:44:20.432392 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.432413 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:20.432565 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:20.433180 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:20.433185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:20.433214 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:20.433418 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.433428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.434641 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.434647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.436210 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.436226 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.438066 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.438072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.438268 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.438304 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:44:20.438400 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 02:44:20.438536 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":20}
I20260808 02:44:20.438876 739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.432577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:33687 (local address 127.25.137.193:46403)
0808 02:44:20.432793 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.432798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.432822 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:44:20.432929 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.432934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.433037 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.433136 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:20.433622 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.434513 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.436368 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.437697 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.437848 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.437919 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.438430 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.438560 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:44:20.439751 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.432424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:33789 (local address 127.25.137.194:34677)
0808 02:44:20.432619 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.432625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.433178 (+ 553us) server_negotiation.cc:407] Connection header received
0808 02:44:20.433564 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.433569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.433642 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.433719 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:20.434305 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.435280 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.437477 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.438671 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.438810 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.438876 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.439389 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.439518 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:44:20.439868 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.432274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:34677 (local address 127.25.137.195:33789)
0808 02:44:20.433076 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.433093 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:20.433304 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:20.433791 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:20.433805 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:20.433830 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:20.434135 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.434145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.435452 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.435456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.437283 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.437299 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.439012 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.439019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.439189 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.439220 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:44:20.439346 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 02:44:20.439491 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":1,"thread_start_us":500,"threads_started":1}
I20260808 02:44:20.440263 763 leader_election.cc:288] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61683f87c863475ab5cca82ec0179317 (127.25.137.195:46371), 330a40e751c44513a3d7110d8724671b (127.25.137.194:34677)
I20260808 02:44:20.441251 457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662acda0d0ea4ca0a0b7ff68155f2d18" candidate_uuid: "61683f87c863475ab5cca82ec0179317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "330a40e751c44513a3d7110d8724671b" is_pre_election: true
I20260808 02:44:20.441483 32728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662acda0d0ea4ca0a0b7ff68155f2d18" candidate_uuid: "61683f87c863475ab5cca82ec0179317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e517527f3145c4a933aee1e5c6c830" is_pre_election: true
I20260808 02:44:20.441780 457 raft_consensus.cc:2454] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61683f87c863475ab5cca82ec0179317 in term 0.
I20260808 02:44:20.441993 32728 raft_consensus.cc:2454] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61683f87c863475ab5cca82ec0179317 in term 0.
W20260808 02:44:20.442613 330 tablet.cc:2369] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:44:20.442638 549 leader_election.cc:302] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 330a40e751c44513a3d7110d8724671b, 61683f87c863475ab5cca82ec0179317; no voters:
I20260808 02:44:20.443058 764 raft_consensus.cc:2781] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:44:20.443192 764 raft_consensus.cc:482] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:44:20.443279 764 raft_consensus.cc:3037] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:20.447293 764 raft_consensus.cc:504] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 } } peers { permanent_uuid: "61683f87c863475ab5cca82ec0179317" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46371 } } peers { permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 } }
I20260808 02:44:20.447980 745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.439609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:58625 (local address 127.25.137.195:46371)
0808 02:44:20.439786 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.439792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.440991 (+ 1199us) server_negotiation.cc:407] Connection header received
0808 02:44:20.441291 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.441296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.441364 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.441425 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:20.442295 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.443311 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.445767 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.447024 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.447173 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.447246 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.447628 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.447756 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:44:20.448144 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.439271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:46371 (local address 127.25.137.193:58625)
0808 02:44:20.440892 (+ 1621us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.440915 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:20.441123 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:20.441629 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:20.441634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:20.441661 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:20.442124 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.442138 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.443501 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.443505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.445590 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.445606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.447221 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.447227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.447406 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.447442 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:44:20.447585 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 02:44:20.447710 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1482,"thread_start_us":5989,"threads_started":1}
I20260808 02:44:20.448410 32728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662acda0d0ea4ca0a0b7ff68155f2d18" candidate_uuid: "61683f87c863475ab5cca82ec0179317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e517527f3145c4a933aee1e5c6c830"
I20260808 02:44:20.448642 32728 raft_consensus.cc:3037] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:20.448868 764 leader_election.cc:288] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [CANDIDATE]: Term 1 election: Requested vote from peers 80e517527f3145c4a933aee1e5c6c830 (127.25.137.193:46403), 330a40e751c44513a3d7110d8724671b (127.25.137.194:34677)
I20260808 02:44:20.449038 651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662acda0d0ea4ca0a0b7ff68155f2d18" candidate_uuid: "80e517527f3145c4a933aee1e5c6c830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61683f87c863475ab5cca82ec0179317" is_pre_election: true
I20260808 02:44:20.449573 651 raft_consensus.cc:2379] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80e517527f3145c4a933aee1e5c6c830 in current term 1: Already voted for candidate 61683f87c863475ab5cca82ec0179317 in this term.
I20260808 02:44:20.450891 457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662acda0d0ea4ca0a0b7ff68155f2d18" candidate_uuid: "61683f87c863475ab5cca82ec0179317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "330a40e751c44513a3d7110d8724671b"
I20260808 02:44:20.451126 457 raft_consensus.cc:3037] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:20.451995 32728 raft_consensus.cc:2454] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61683f87c863475ab5cca82ec0179317 in term 1.
I20260808 02:44:20.452461 554 leader_election.cc:302] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61683f87c863475ab5cca82ec0179317, 80e517527f3145c4a933aee1e5c6c830; no voters:
I20260808 02:44:20.452678 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.439528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:34677 (local address 127.25.137.193:50067)
0808 02:44:20.445594 (+ 6066us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.445612 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:20.445753 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:20.446078 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:20.446082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:20.446108 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:20.446465 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.446478 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.447838 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.447842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.450536 (+ 2694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:20.450551 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.451946 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.451953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:20.452106 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.452139 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:44:20.452223 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 02:44:20.452350 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5914,"thread_start_us":1019,"threads_started":1}
I20260808 02:44:20.452771 773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.440056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:50067 (local address 127.25.137.194:34677)
0808 02:44:20.445076 (+ 5020us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.445082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.445859 (+ 777us) server_negotiation.cc:407] Connection header received
0808 02:44:20.445909 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.445912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.445963 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.446040 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:20.446653 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.447678 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.450689 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.451727 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.451919 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.451993 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.452291 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.452451 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4862,"thread_start_us":6336,"threads_started":1}
I20260808 02:44:20.453050 764 raft_consensus.cc:2781] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:20.453383 456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662acda0d0ea4ca0a0b7ff68155f2d18" candidate_uuid: "80e517527f3145c4a933aee1e5c6c830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "330a40e751c44513a3d7110d8724671b" is_pre_election: true
I20260808 02:44:20.453539 764 raft_consensus.cc:686] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 1 LEADER]: Becoming Leader. State: Replica: 61683f87c863475ab5cca82ec0179317, State: Running, Role: LEADER
I20260808 02:44:20.454284 32622 leader_election.cc:302] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80e517527f3145c4a933aee1e5c6c830; no voters: 330a40e751c44513a3d7110d8724671b, 61683f87c863475ab5cca82ec0179317
I20260808 02:44:20.454373 457 raft_consensus.cc:2454] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61683f87c863475ab5cca82ec0179317 in term 1.
I20260808 02:44:20.454391 764 consensus_queue.cc:237] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 } } peers { permanent_uuid: "61683f87c863475ab5cca82ec0179317" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46371 } } peers { permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 } }
I20260808 02:44:20.454640 763 raft_consensus.cc:2726] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 02:44:20.458969 32492 catalog_manager.cc:5495] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61683f87c863475ab5cca82ec0179317 (127.25.137.195). New cstate: current_term: 1 leader_uuid: "61683f87c863475ab5cca82ec0179317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61683f87c863475ab5cca82ec0179317" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 } health_report { overall_health: UNKNOWN } } }
I20260808 02:44:20.503652 745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.498380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57890 (local address 127.25.137.195:46371)
0808 02:44:20.498561 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.498566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.498585 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:44:20.498693 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.498697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.498829 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.498965 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:20.499333 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.500158 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.501546 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.501951 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.502061 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.502540 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:20.503049 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:20.503180 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.503284 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.503371 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:44:20.511906 457 raft_consensus.cc:1261] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b [term 1 FOLLOWER]: Refusing update from remote peer 61683f87c863475ab5cca82ec0179317: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:20.511960 32728 raft_consensus.cc:1261] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 1 FOLLOWER]: Refusing update from remote peer 61683f87c863475ab5cca82ec0179317: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:20.512917 775 consensus_queue.cc:1035] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "330a40e751c44513a3d7110d8724671b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:20.513346 764 consensus_queue.cc:1035] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80e517527f3145c4a933aee1e5c6c830" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:20.525246 780 mvcc.cc:205] Tried to move back new op lower bound from 7316099319846359040 to 7316099319625252864. Current Snapshot: MvccSnapshot[applied={T|T < 7316099319846359040}]
I20260808 02:44:20.527114 778 mvcc.cc:205] Tried to move back new op lower bound from 7316099319846359040 to 7316099319625252864. Current Snapshot: MvccSnapshot[applied={T|T < 7316099319846359040 or (T in {7316099319846359040})}]
I20260808 02:44:20.537544 32492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:20.537797 32492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:20.541236 32492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08d1db3e1ccf49ab80e255a63dd90d71: Sending DeleteTablet for 3 replicas of tablet 662acda0d0ea4ca0a0b7ff68155f2d18
I20260808 02:44:20.542434 623 tablet_service.cc:1514] Processing DeleteTablet for tablet 662acda0d0ea4ca0a0b7ff68155f2d18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:20 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:57874
I20260808 02:44:20.542855 428 tablet_service.cc:1514] Processing DeleteTablet for tablet 662acda0d0ea4ca0a0b7ff68155f2d18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:20 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:39896
I20260808 02:44:20.542878 32697 tablet_service.cc:1514] Processing DeleteTablet for tablet 662acda0d0ea4ca0a0b7ff68155f2d18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:20 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:56062
I20260808 02:44:20.545058 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 32602
I20260808 02:44:20.554288 790 tablet_replica.cc:330] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: stopping tablet replica
I20260808 02:44:20.554749 790 raft_consensus.cc:2229] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:20.555310 790 raft_consensus.cc:2258] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:20.559310 790 ts_tablet_manager.cc:1905] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 02:44:20 dist-test-slave-k5rr krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:44:20.564211 789 tablet_replica.cc:330] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: stopping tablet replica
I20260808 02:44:20.564718 789 raft_consensus.cc:2229] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:20.565290 789 raft_consensus.cc:2258] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:20.571099 789 ts_tablet_manager.cc:1905] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:20.571767 788 tablet_replica.cc:330] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: stopping tablet replica
I20260808 02:44:20.572342 788 raft_consensus.cc:2229] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:44:20.573112 788 raft_consensus.cc:2258] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:20.573648 739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.546593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.25.137.193:46403)
0808 02:44:20.546778 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.546784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.551388 (+ 4604us) server_negotiation.cc:407] Connection header received
0808 02:44:20.551596 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.551602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.551687 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.551799 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:20.555701 (+ 3902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.556899 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.558750 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.559481 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.559580 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.565345 (+ 5765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:20.565373 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:20.565392 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:20.565438 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:20.569400 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:20.570317 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:20.570326 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:20.570329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:20.570497 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:20.570971 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:20.570977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:20.570981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:20.571454 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:20.571774 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.572431 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.573221 (+ 790us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:44:20.574714 322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:20.581473 789 ts_tablet_manager.cc:1918] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:20.581669 789 log.cc:1196] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/662acda0d0ea4ca0a0b7ff68155f2d18
I20260808 02:44:20.582231 789 ts_tablet_manager.cc:1939] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 330a40e751c44513a3d7110d8724671b: Deleting consensus metadata
I20260808 02:44:20.582696 788 ts_tablet_manager.cc:1905] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:20.584164 32473 catalog_manager.cc:4841] TS 330a40e751c44513a3d7110d8724671b (127.25.137.194:34677): tablet 662acda0d0ea4ca0a0b7ff68155f2d18 (table test-table [id=142e58da73fb42dab5d849c30ef5b2b2]) successfully deleted
I20260808 02:44:20.593362 788 ts_tablet_manager.cc:1918] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:20.593573 788 log.cc:1196] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/662acda0d0ea4ca0a0b7ff68155f2d18
I20260808 02:44:20.594133 788 ts_tablet_manager.cc:1939] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 61683f87c863475ab5cca82ec0179317: Deleting consensus metadata
I20260808 02:44:20.596141 32476 catalog_manager.cc:4841] TS 61683f87c863475ab5cca82ec0179317 (127.25.137.195:46371): tablet 662acda0d0ea4ca0a0b7ff68155f2d18 (table test-table [id=142e58da73fb42dab5d849c30ef5b2b2]) successfully deleted
I20260808 02:44:20.676846 790 ts_tablet_manager.cc:1918] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:20.677163 790 log.cc:1196] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/662acda0d0ea4ca0a0b7ff68155f2d18
I20260808 02:44:20.677584 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 32602
I20260808 02:44:20.677621 790 ts_tablet_manager.cc:1939] T 662acda0d0ea4ca0a0b7ff68155f2d18 P 80e517527f3145c4a933aee1e5c6c830: Deleting consensus metadata
W20260808 02:44:20.699429 32476 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.137.193:46403 (error 108)
I20260808 02:44:20.700392 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 337
Aug 08 02:44:20 dist-test-slave-k5rr krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:44:20.712798 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.712200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46403 (local address 127.0.0.1:56070)
0808 02:44:20.712392 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.712614 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:46403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:44:20.713166 32476 catalog_manager.cc:4584] TS 80e517527f3145c4a933aee1e5c6c830 (127.25.137.193:46403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 662acda0d0ea4ca0a0b7ff68155f2d18: Network error: Client connection negotiation failed: client connection to 127.25.137.193:46403: connect: Connection refused (error 111)
I20260808 02:44:20.719502 773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.702158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39906 (local address 127.25.137.194:34677)
0808 02:44:20.702377 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.702384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.702405 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:44:20.702473 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.702479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.702561 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.702670 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:20.704602 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.705893 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.707953 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.708666 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.708786 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.713947 (+ 5161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:20.713960 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:20.713976 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:20.714014 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:20.716773 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:20.717457 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:20.717464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:20.717466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:20.717612 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:20.717980 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:20.717985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:20.717989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:20.718362 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:20.718638 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.718944 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.719170 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:44:20.720167 506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:20.767266 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.766688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46403 (local address 127.0.0.1:56080)
0808 02:44:20.766953 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.767068 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:46403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 02:44:20.768584 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.768085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46403 (local address 127.0.0.1:56082)
0808 02:44:20.768282 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.768406 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:46403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 02:44:20.811359 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 337
I20260808 02:44:20.829982 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.829385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46403 (local address 127.0.0.1:56092)
0808 02:44:20.829655 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.829798 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:46403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 02:44:20.831269 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.830841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46403 (local address 127.0.0.1:56098)
0808 02:44:20.831013 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.831111 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:46403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 02:44:20.835815 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 524
Aug 08 02:44:20 dist-test-slave-k5rr krb5kdc[32436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:44:20.852538 745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.836653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57896 (local address 127.25.137.195:46371)
0808 02:44:20.836907 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:20.836914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:20.836935 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:44:20.837034 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:20.837041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:20.837140 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:20.837242 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:20.838618 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.839441 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.840730 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:20.841405 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:20.841534 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:20.846003 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:20.846018 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:20.846035 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:20.846087 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:20.849528 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:20.850248 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:20.850256 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:20.850258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:20.850405 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:20.850828 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:20.850832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:20.850834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:20.851270 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:20.851626 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:20.851956 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 02:44:20.852197 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:44:20.853338 705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:20.911916 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.911415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46403 (local address 127.0.0.1:56110)
0808 02:44:20.911632 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.911727 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:46403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 02:44:20.913118 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:20.912665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46403 (local address 127.0.0.1:56126)
0808 02:44:20.912886 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:20.912962 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:46403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260808 02:44:20.977941 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 524
I20260808 02:44:21.004526 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 32452
I20260808 02:44:21.005858 32546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:21.117152 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:21.116692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46403 (local address 127.0.0.1:56142)
0808 02:44:21.116863 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:21.116948 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:46403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 02:44:21.117236 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 32452
2026-08-08T02:44:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4446 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T02:44:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:21Z Disabled control of system clock
I20260808 02:44:21.195078 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:35411
--webserver_interface=127.25.137.254
--webserver_port=0
--builtin_ntp_servers=127.25.137.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:35411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:44:21.518110 803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:21.518384 803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:21.518484 803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:21.525858 803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:21.525977 803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:21.526031 803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:21.526089 803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:21.526134 803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:21.535334 803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:35411
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:35411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:21.536557 803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:21.538344 803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:21.549494 812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:21.549484 814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:21.549506 817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:21.550519 803 server_base.cc:1034] running on GCE node
I20260808 02:44:21.551362 803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:21.552747 803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:21.553944 803 hybrid_clock.cc:648] HybridClock initialized: now 1786157061553871 us; error 57 us; skew 500 ppm
I20260808 02:44:21.554282 803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:44:21.556444 803 webserver.cc:460] Webserver started at http://127.25.137.254:41903/ using document root <none> and password file <none>
I20260808 02:44:21.557158 803 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:21.557251 803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:21.557555 803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:21.560350 803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "b59cf1d27bdf4812aaf7243b8a77a503"
format_stamp: "Formatted at 2026-08-08 02:44:21 on dist-test-slave-k5rr"
server_key: "fa80c5a16deb7784c0e2e6421dddf9f4"
server_key_iv: "381ffeb22c94a5a6757347000453c120"
server_key_version: "encryptionkey@0"
I20260808 02:44:21.561179 803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "b59cf1d27bdf4812aaf7243b8a77a503"
format_stamp: "Formatted at 2026-08-08 02:44:21 on dist-test-slave-k5rr"
server_key: "fa80c5a16deb7784c0e2e6421dddf9f4"
server_key_iv: "381ffeb22c94a5a6757347000453c120"
server_key_version: "encryptionkey@0"
I20260808 02:44:21.566160 803 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:44:21.569494 825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:21.570756 803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:21.570905 803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "b59cf1d27bdf4812aaf7243b8a77a503"
format_stamp: "Formatted at 2026-08-08 02:44:21 on dist-test-slave-k5rr"
server_key: "fa80c5a16deb7784c0e2e6421dddf9f4"
server_key_iv: "381ffeb22c94a5a6757347000453c120"
server_key_version: "encryptionkey@0"
I20260808 02:44:21.571113 803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:21.576721 803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:21.577471 803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:21.577739 803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:21.594924 803 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:35411
I20260808 02:44:21.594956 908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:35411 every 8 connection(s)
I20260808 02:44:21.596235 803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:21.600098 909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:21.603915 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 803
I20260808 02:44:21.604138 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:21.604959 26151 external_mini_cluster.cc:1428] Setting key d0aaef8b47c15daeeac8cc6837f7d3de
I20260808 02:44:21.613130 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:21.607039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.25.137.254:35411)
0808 02:44:21.608039 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:21.608049 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:21.608088 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:44:21.609675 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:21.609697 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:21.610109 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:21.610491 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:21.611519 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:21.611528 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:21.611546 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:21.611562 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:21.611621 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:21.611637 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:21.611848 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:21.611989 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 02:44:21.612231 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":349,"threads_started":1}
I20260808 02:44:21.613205 909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503: Bootstrap starting.
I20260808 02:44:21.616251 909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:21.617265 909 log.cc:824] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:21.620592 909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503: No bootstrap required, opened a new log
I20260808 02:44:21.628309 909 raft_consensus.cc:348] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59cf1d27bdf4812aaf7243b8a77a503" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 35411 } }
I20260808 02:44:21.628664 909 raft_consensus.cc:374] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:21.628757 909 raft_consensus.cc:729] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b59cf1d27bdf4812aaf7243b8a77a503, State: Initialized, Role: FOLLOWER
I20260808 02:44:21.629531 909 consensus_queue.cc:260] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59cf1d27bdf4812aaf7243b8a77a503" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 35411 } }
I20260808 02:44:21.629860 916 raft_consensus.cc:482] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:21.630095 916 raft_consensus.cc:504] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59cf1d27bdf4812aaf7243b8a77a503" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 35411 } }
I20260808 02:44:21.629797 909 raft_consensus.cc:388] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:21.630699 909 raft_consensus.cc:482] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:21.630817 909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:21.630656 916 leader_election.cc:302] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b59cf1d27bdf4812aaf7243b8a77a503; no voters:
I20260808 02:44:21.631405 916 leader_election.cc:288] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260808 02:44:21.633462 909 raft_consensus.cc:504] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59cf1d27bdf4812aaf7243b8a77a503" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 35411 } }
I20260808 02:44:21.633674 909 leader_election.cc:302] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b59cf1d27bdf4812aaf7243b8a77a503; no voters:
I20260808 02:44:21.633733 917 raft_consensus.cc:2741] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260808 02:44:21.633873 909 leader_election.cc:288] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:21.634084 916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:21.635249 916 raft_consensus.cc:686] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [term 1 LEADER]: Becoming Leader. State: Replica: b59cf1d27bdf4812aaf7243b8a77a503, State: Running, Role: LEADER
I20260808 02:44:21.635829 916 consensus_queue.cc:237] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59cf1d27bdf4812aaf7243b8a77a503" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 35411 } }
I20260808 02:44:21.636860 909 sys_catalog.cc:564] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:21.637856 921 sys_catalog.cc:455] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b59cf1d27bdf4812aaf7243b8a77a503. Latest consensus state: current_term: 1 leader_uuid: "b59cf1d27bdf4812aaf7243b8a77a503" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59cf1d27bdf4812aaf7243b8a77a503" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 35411 } } }
I20260808 02:44:21.638293 921 sys_catalog.cc:458] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:21.637898 917 sys_catalog.cc:455] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b59cf1d27bdf4812aaf7243b8a77a503" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59cf1d27bdf4812aaf7243b8a77a503" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 35411 } } }
I20260808 02:44:21.638706 917 sys_catalog.cc:458] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:21.640077 926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:21.645252 926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:21.654155 926 catalog_manager.cc:1347] Generated new cluster ID: 9277dd8798d440f3971d566d7ba67e10
I20260808 02:44:21.654271 926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:21.673799 926 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:21.674230 926 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:21.680823 926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503: Generated new TSK 0
I20260808 02:44:21.681612 926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:44:21.687953 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:35411
--builtin_ntp_servers=127.25.137.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:44:22.021469 946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:22.021734 946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:22.021878 946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:22.029049 946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:22.029156 946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:22.029323 946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:44:22.038321 946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:35411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:22.039543 946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:22.041397 946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:22.054025 957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:22.054203 954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:22.054085 953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:22.055114 946 server_base.cc:1034] running on GCE node
I20260808 02:44:22.055694 946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:22.056625 946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:22.057826 946 hybrid_clock.cc:648] HybridClock initialized: now 1786157062057767 us; error 54 us; skew 500 ppm
I20260808 02:44:22.058135 946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:44:22.060339 946 webserver.cc:460] Webserver started at http://127.25.137.193:39627/ using document root <none> and password file <none>
I20260808 02:44:22.060986 946 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:22.061128 946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:22.061460 946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:22.064175 946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "35af09257a104185a0ed3682ff9a66ce"
format_stamp: "Formatted at 2026-08-08 02:44:22 on dist-test-slave-k5rr"
server_key: "58ad12524956af6e5eae9f94c75d4e98"
server_key_iv: "8650dba5d49fdde8e197e72f4228ec21"
server_key_version: "encryptionkey@0"
I20260808 02:44:22.064956 946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "35af09257a104185a0ed3682ff9a66ce"
format_stamp: "Formatted at 2026-08-08 02:44:22 on dist-test-slave-k5rr"
server_key: "58ad12524956af6e5eae9f94c75d4e98"
server_key_iv: "8650dba5d49fdde8e197e72f4228ec21"
server_key_version: "encryptionkey@0"
I20260808 02:44:22.070022 946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:44:22.073396 965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:22.074730 946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 02:44:22.074885 946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "35af09257a104185a0ed3682ff9a66ce"
format_stamp: "Formatted at 2026-08-08 02:44:22 on dist-test-slave-k5rr"
server_key: "58ad12524956af6e5eae9f94c75d4e98"
server_key_iv: "8650dba5d49fdde8e197e72f4228ec21"
server_key_version: "encryptionkey@0"
I20260808 02:44:22.075084 946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:22.080758 946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:22.081589 946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:22.081844 946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:22.082806 946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:22.084156 946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:22.084259 946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:22.084369 946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:22.084439 946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:22.115211 946 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:40221
I20260808 02:44:22.115267 1124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:40221 every 8 connection(s)
I20260808 02:44:22.116519 946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:44:22.121553 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 946
I20260808 02:44:22.121810 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:44:22.122463 26151 external_mini_cluster.cc:1428] Setting key 72873878637c85447484b5beed7764b2
I20260808 02:44:22.125293 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:22.119175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:37951 (local address 127.25.137.254:35411)
0808 02:44:22.119652 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:22.119660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:22.120854 (+ 1194us) server_negotiation.cc:407] Connection header received
0808 02:44:22.122712 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:22.122719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:22.122790 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:22.122929 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:22.124363 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:22.124370 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:22.124374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:22.124397 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:22.124456 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:22.124474 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:22.124579 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:22.124868 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 02:44:22.124936 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":218,"threads_started":1}
I20260808 02:44:22.125785 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:35411
--builtin_ntp_servers=127.25.137.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 02:44:22.126097 1129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:22.119788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:35411 (local address 127.25.137.193:37951)
0808 02:44:22.120689 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:22.120734 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:22.122389 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:22.123109 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:22.123120 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:22.123248 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:22.124001 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:44:22.124005 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:22.124054 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:44:22.124159 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:22.124583 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:22.124595 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:22.124603 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:44:22.124888 (+ 285us) client_negotiation.cc:240] Negotiation successful
0808 02:44:22.125126 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":281,"threads_started":1}
I20260808 02:44:22.127595 1125 heartbeater.cc:347] Connected to a master server at 127.25.137.254:35411
I20260808 02:44:22.127951 1125 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:22.128667 1125 heartbeater.cc:511] Master 127.25.137.254:35411 requested a full tablet report, sending...
I20260808 02:44:22.130908 858 ts_manager.cc:194] Registered new tserver with Master: 35af09257a104185a0ed3682ff9a66ce (127.25.137.193:40221)
W20260808 02:44:22.452085 1130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:22.452355 1130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:22.452473 1130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:22.459779 1130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:22.459878 1130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:22.460011 1130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:44:22.469961 1130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:35411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:22.471218 1130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:22.472981 1130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:22.485688 1143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:22.485824 1151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:22.485901 1147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:22.486752 1130 server_base.cc:1034] running on GCE node
I20260808 02:44:22.487325 1130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:22.488212 1130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:22.489418 1130 hybrid_clock.cc:648] HybridClock initialized: now 1786157062489366 us; error 46 us; skew 500 ppm
I20260808 02:44:22.489707 1130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:44:22.491863 1130 webserver.cc:460] Webserver started at http://127.25.137.194:35967/ using document root <none> and password file <none>
I20260808 02:44:22.492512 1130 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:22.492612 1130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:22.492924 1130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:22.495720 1130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "ce9adb4528e649fdb82682d8036bc7e7"
format_stamp: "Formatted at 2026-08-08 02:44:22 on dist-test-slave-k5rr"
server_key: "ee50cf0b276417fad27a22508f1b8d8d"
server_key_iv: "5faaad096374550c7daec068ffcba999"
server_key_version: "encryptionkey@0"
I20260808 02:44:22.496519 1130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "ce9adb4528e649fdb82682d8036bc7e7"
format_stamp: "Formatted at 2026-08-08 02:44:22 on dist-test-slave-k5rr"
server_key: "ee50cf0b276417fad27a22508f1b8d8d"
server_key_iv: "5faaad096374550c7daec068ffcba999"
server_key_version: "encryptionkey@0"
I20260808 02:44:22.501608 1130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 02:44:22.505080 1162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:22.506362 1130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:22.506510 1130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "ce9adb4528e649fdb82682d8036bc7e7"
format_stamp: "Formatted at 2026-08-08 02:44:22 on dist-test-slave-k5rr"
server_key: "ee50cf0b276417fad27a22508f1b8d8d"
server_key_iv: "5faaad096374550c7daec068ffcba999"
server_key_version: "encryptionkey@0"
I20260808 02:44:22.506716 1130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:22.512214 1130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:22.512982 1130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:22.513258 1130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:22.514139 1130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:22.515544 1130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:22.515640 1130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:22.515748 1130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:22.515816 1130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:22.547384 1130 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:33035
I20260808 02:44:22.547453 1340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:33035 every 8 connection(s)
I20260808 02:44:22.548709 1130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:44:22.550112 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 1130
I20260808 02:44:22.550288 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:44:22.550722 26151 external_mini_cluster.cc:1428] Setting key c47ae5210d4e3dd0f850087aa531a7a7
I20260808 02:44:22.553798 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:35411
--builtin_ntp_servers=127.25.137.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 02:44:22.557209 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:22.551311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:37761 (local address 127.25.137.254:35411)
0808 02:44:22.551541 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:22.551546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:22.553227 (+ 1681us) server_negotiation.cc:407] Connection header received
0808 02:44:22.555057 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:22.555062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:22.555125 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:22.555209 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:22.556460 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:22.556466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:22.556471 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:22.556490 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:22.556520 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:22.556531 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:22.556629 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:22.556822 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 02:44:22.556888 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:44:22.558104 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:22.551946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:35411 (local address 127.25.137.194:37761)
0808 02:44:22.553090 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:22.553136 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:22.554784 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:22.555448 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:22.555456 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:22.555542 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:22.556167 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:44:22.556169 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:22.556210 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:44:22.556294 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:22.556614 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:22.556625 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:22.556632 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:44:22.556899 (+ 267us) client_negotiation.cc:240] Negotiation successful
0808 02:44:22.557185 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":311,"threads_started":1}
I20260808 02:44:22.559378 1341 heartbeater.cc:347] Connected to a master server at 127.25.137.254:35411
I20260808 02:44:22.559680 1341 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:22.560331 1341 heartbeater.cc:511] Master 127.25.137.254:35411 requested a full tablet report, sending...
I20260808 02:44:22.561851 858 ts_manager.cc:194] Registered new tserver with Master: ce9adb4528e649fdb82682d8036bc7e7 (127.25.137.194:33035)
W20260808 02:44:22.891773 1347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:22.892076 1347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:22.892215 1347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:22.899525 1347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:22.899634 1347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:22.899772 1347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:44:22.908849 1347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:35411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:22.910187 1347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:22.912061 1347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:22.925066 1353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:22.925109 1354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:22.925130 1356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:22.925993 1347 server_base.cc:1034] running on GCE node
I20260808 02:44:22.926568 1347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:22.927438 1347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:22.928627 1347 hybrid_clock.cc:648] HybridClock initialized: now 1786157062928571 us; error 51 us; skew 500 ppm
I20260808 02:44:22.928920 1347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:44:22.931083 1347 webserver.cc:460] Webserver started at http://127.25.137.195:46523/ using document root <none> and password file <none>
I20260808 02:44:22.931777 1347 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:22.931921 1347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:22.932257 1347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:22.935094 1347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "527587269ecf4bd2b56c320de373796a"
format_stamp: "Formatted at 2026-08-08 02:44:22 on dist-test-slave-k5rr"
server_key: "f52980e977576a7277f4b94b24321a23"
server_key_iv: "ff89a2a36e5703a38f54264dd083f1f7"
server_key_version: "encryptionkey@0"
I20260808 02:44:22.935890 1347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "527587269ecf4bd2b56c320de373796a"
format_stamp: "Formatted at 2026-08-08 02:44:22 on dist-test-slave-k5rr"
server_key: "f52980e977576a7277f4b94b24321a23"
server_key_iv: "ff89a2a36e5703a38f54264dd083f1f7"
server_key_version: "encryptionkey@0"
I20260808 02:44:22.940980 1347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 02:44:22.944375 1367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:22.945734 1347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:44:22.945894 1347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "527587269ecf4bd2b56c320de373796a"
format_stamp: "Formatted at 2026-08-08 02:44:22 on dist-test-slave-k5rr"
server_key: "f52980e977576a7277f4b94b24321a23"
server_key_iv: "ff89a2a36e5703a38f54264dd083f1f7"
server_key_version: "encryptionkey@0"
I20260808 02:44:22.946162 1347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:22.951750 1347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:22.952548 1347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:22.952809 1347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:22.953794 1347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:22.955281 1347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:22.955387 1347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:22.955497 1347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:22.955566 1347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:22.985827 1347 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:34961
I20260808 02:44:22.985899 1538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:34961 every 8 connection(s)
I20260808 02:44:22.987147 1347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:44:22.987690 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 1347
I20260808 02:44:22.987867 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:44:22.988430 26151 external_mini_cluster.cc:1428] Setting key df03aac35d7d40585dde93610e183009
I20260808 02:44:22.994989 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:22.989719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:32839 (local address 127.25.137.254:35411)
0808 02:44:22.989933 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:22.989939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:22.991367 (+ 1428us) server_negotiation.cc:407] Connection header received
0808 02:44:22.993147 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:22.993151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:22.993204 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:22.993288 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:22.994486 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:22.994490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:22.994493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:22.994506 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:22.994533 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:22.994543 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:22.994634 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:22.994751 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 02:44:22.994798 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 02:44:22.995891 1543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:22.990286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:35411 (local address 127.25.137.195:32839)
0808 02:44:22.991216 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:22.991263 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:22.992880 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:22.993479 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:22.993487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:22.993575 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:22.994217 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:44:22.994220 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:22.994256 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:44:22.994344 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:22.994605 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:22.994610 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:22.994614 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:44:22.994822 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 02:44:22.995008 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":286,"threads_started":1}
I20260808 02:44:22.997100 1541 heartbeater.cc:347] Connected to a master server at 127.25.137.254:35411
I20260808 02:44:22.997340 1541 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:22.997872 1541 heartbeater.cc:511] Master 127.25.137.254:35411 requested a full tablet report, sending...
I20260808 02:44:22.999123 858 ts_manager.cc:194] Registered new tserver with Master: 527587269ecf4bd2b56c320de373796a (127.25.137.195:34961)
I20260808 02:44:23.003752 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:44:23.010681 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.008140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40288 (local address 127.25.137.254:35411)
0808 02:44:23.008306 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:23.008310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:23.008818 (+ 508us) server_negotiation.cc:407] Connection header received
0808 02:44:23.008977 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:23.008982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:23.009075 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:23.009164 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:23.010072 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:23.010079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:23.010083 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:23.010103 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:23.010134 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:23.010145 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:23.010248 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:23.010368 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 02:44:23.010425 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:44:23.014645 858 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:44:23.021407 858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:44:23.041709 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.032665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:40221 (local address 127.0.0.1:49298)
0808 02:44:23.034413 (+ 1748us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.034468 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:23.034602 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:23.037049 (+ 2447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:23.037058 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:23.037161 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:23.038132 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:44:23.038136 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:23.038171 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:44:23.038271 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:23.040831 (+ 2560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:23.040845 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:23.040854 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 02:44:23.041074 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 02:44:23.041180 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1457,"thread_start_us":2506,"threads_started":1}
I20260808 02:44:23.042812 1562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.032858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49298 (local address 127.25.137.193:40221)
0808 02:44:23.035152 (+ 2294us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:23.035162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:23.035194 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:44:23.035272 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:23.035281 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:23.035523 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:23.035675 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:23.039331 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:23.039341 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:23.039357 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:23.039374 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:23.040562 (+ 1188us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:23.040610 (+ 48us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:23.040757 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:23.042338 (+ 1581us) server_negotiation.cc:299] Negotiation successful
0808 02:44:23.042471 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2114,"thread_start_us":1980,"threads_started":1}
I20260808 02:44:23.047173 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.033637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40294 (local address 127.25.137.195:34961)
0808 02:44:23.043593 (+ 9956us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:23.043599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:23.043650 (+ 51us) server_negotiation.cc:407] Connection header received
0808 02:44:23.043722 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:23.043729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:23.043994 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:23.044163 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:23.046135 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:23.046144 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:23.046158 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:23.046177 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:23.046232 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:23.046254 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:23.046404 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:23.046677 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 02:44:23.046773 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9794,"thread_start_us":9613,"threads_started":1}
I20260808 02:44:23.047881 1560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.033335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:34961 (local address 127.0.0.1:40294)
0808 02:44:23.035443 (+ 2108us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.035461 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:23.035554 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:23.045141 (+ 9587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:23.045147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:23.045173 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:23.045883 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:44:23.045887 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:23.045904 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:44:23.045952 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:23.046503 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:23.046508 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:23.046511 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:44:23.047471 (+ 960us) client_negotiation.cc:240] Negotiation successful
0808 02:44:23.047525 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1969,"thread_start_us":612,"threads_started":1}
I20260808 02:44:23.048377 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.035788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:33035 (local address 127.0.0.1:55928)
0808 02:44:23.042071 (+ 6283us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.042090 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:23.042173 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:23.046568 (+ 4395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:23.046572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:23.046614 (+ 42us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:23.047116 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:44:23.047119 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:23.047135 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:44:23.047179 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:23.047971 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:23.047975 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:23.047981 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:44:23.048067 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:44:23.048118 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6167,"thread_start_us":9446,"threads_started":1}
I20260808 02:44:23.048501 1567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.035893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55928 (local address 127.25.137.194:33035)
0808 02:44:23.046098 (+ 10205us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:23.046106 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:23.046133 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:44:23.046197 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:23.046204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:23.046408 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:23.046547 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:23.047776 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:23.047781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:23.047800 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:23.047817 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:23.047851 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:23.047864 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:23.048016 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:23.048084 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 02:44:23.048170 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10027,"thread_start_us":9633,"threads_started":1}
I20260808 02:44:23.049242 1030 tablet_service.cc:1467] Processing CreateTablet for tablet 2f8e9b260486461ca9b845ed089bdca9 (DEFAULT_TABLE table=test-table [id=c371efa89ce5493eb40b6b0f1afca941]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:23.050889 1030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f8e9b260486461ca9b845ed089bdca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:23.051855 1442 tablet_service.cc:1467] Processing CreateTablet for tablet 2f8e9b260486461ca9b845ed089bdca9 (DEFAULT_TABLE table=test-table [id=c371efa89ce5493eb40b6b0f1afca941]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:23.052099 1239 tablet_service.cc:1467] Processing CreateTablet for tablet 2f8e9b260486461ca9b845ed089bdca9 (DEFAULT_TABLE table=test-table [id=c371efa89ce5493eb40b6b0f1afca941]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:23.053274 1442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f8e9b260486461ca9b845ed089bdca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:23.053274 1239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f8e9b260486461ca9b845ed089bdca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:23.064580 1569 tablet_bootstrap.cc:492] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7: Bootstrap starting.
I20260808 02:44:23.067595 1569 tablet_bootstrap.cc:654] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:23.068581 1569 log.cc:824] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:23.069540 1570 tablet_bootstrap.cc:492] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce: Bootstrap starting.
I20260808 02:44:23.071997 1572 tablet_bootstrap.cc:492] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: Bootstrap starting.
I20260808 02:44:23.072997 1569 tablet_bootstrap.cc:492] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7: No bootstrap required, opened a new log
I20260808 02:44:23.073287 1569 ts_tablet_manager.cc:1397] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260808 02:44:23.073505 1570 tablet_bootstrap.cc:654] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:23.074808 1570 log.cc:824] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:23.075673 1572 tablet_bootstrap.cc:654] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:23.076928 1572 log.cc:824] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:23.083716 1569 raft_consensus.cc:348] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35af09257a104185a0ed3682ff9a66ce" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 40221 } } peers { permanent_uuid: "ce9adb4528e649fdb82682d8036bc7e7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33035 } } peers { permanent_uuid: "527587269ecf4bd2b56c320de373796a" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34961 } }
I20260808 02:44:23.084189 1569 raft_consensus.cc:374] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:23.084285 1569 raft_consensus.cc:729] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9adb4528e649fdb82682d8036bc7e7, State: Initialized, Role: FOLLOWER
I20260808 02:44:23.085064 1569 consensus_queue.cc:260] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35af09257a104185a0ed3682ff9a66ce" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 40221 } } peers { permanent_uuid: "ce9adb4528e649fdb82682d8036bc7e7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33035 } } peers { permanent_uuid: "527587269ecf4bd2b56c320de373796a" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34961 } }
I20260808 02:44:23.086845 1341 heartbeater.cc:503] Master 127.25.137.254:35411 was elected leader, sending a full tablet report...
I20260808 02:44:23.087911 1569 ts_tablet_manager.cc:1428] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260808 02:44:23.088871 1572 tablet_bootstrap.cc:492] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: No bootstrap required, opened a new log
I20260808 02:44:23.089159 1572 ts_tablet_manager.cc:1397] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.000s
I20260808 02:44:23.090994 1570 tablet_bootstrap.cc:492] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce: No bootstrap required, opened a new log
I20260808 02:44:23.091289 1570 ts_tablet_manager.cc:1397] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce: Time spent bootstrapping tablet: real 0.022s user 0.016s sys 0.004s
I20260808 02:44:23.097263 1572 raft_consensus.cc:348] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35af09257a104185a0ed3682ff9a66ce" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 40221 } } peers { permanent_uuid: "ce9adb4528e649fdb82682d8036bc7e7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33035 } } peers { permanent_uuid: "527587269ecf4bd2b56c320de373796a" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34961 } }
I20260808 02:44:23.097618 1572 raft_consensus.cc:374] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:23.097718 1572 raft_consensus.cc:729] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 527587269ecf4bd2b56c320de373796a, State: Initialized, Role: FOLLOWER
I20260808 02:44:23.098346 1572 consensus_queue.cc:260] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35af09257a104185a0ed3682ff9a66ce" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 40221 } } peers { permanent_uuid: "ce9adb4528e649fdb82682d8036bc7e7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33035 } } peers { permanent_uuid: "527587269ecf4bd2b56c320de373796a" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34961 } }
I20260808 02:44:23.098909 1570 raft_consensus.cc:348] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35af09257a104185a0ed3682ff9a66ce" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 40221 } } peers { permanent_uuid: "ce9adb4528e649fdb82682d8036bc7e7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33035 } } peers { permanent_uuid: "527587269ecf4bd2b56c320de373796a" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34961 } }
I20260808 02:44:23.099337 1570 raft_consensus.cc:374] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:23.099438 1570 raft_consensus.cc:729] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35af09257a104185a0ed3682ff9a66ce, State: Initialized, Role: FOLLOWER
I20260808 02:44:23.099491 1541 heartbeater.cc:503] Master 127.25.137.254:35411 was elected leader, sending a full tablet report...
I20260808 02:44:23.100072 1572 ts_tablet_manager.cc:1428] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260808 02:44:23.100040 1570 consensus_queue.cc:260] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35af09257a104185a0ed3682ff9a66ce" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 40221 } } peers { permanent_uuid: "ce9adb4528e649fdb82682d8036bc7e7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33035 } } peers { permanent_uuid: "527587269ecf4bd2b56c320de373796a" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34961 } }
I20260808 02:44:23.101156 1125 heartbeater.cc:503] Master 127.25.137.254:35411 was elected leader, sending a full tablet report...
I20260808 02:44:23.101661 1570 ts_tablet_manager.cc:1428] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
W20260808 02:44:23.118795 1126 tablet.cc:2369] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:44:23.199718 1581 raft_consensus.cc:482] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:23.200062 1581 raft_consensus.cc:504] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35af09257a104185a0ed3682ff9a66ce" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 40221 } } peers { permanent_uuid: "ce9adb4528e649fdb82682d8036bc7e7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33035 } } peers { permanent_uuid: "527587269ecf4bd2b56c320de373796a" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34961 } }
I20260808 02:44:23.201901 1581 leader_election.cc:288] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce9adb4528e649fdb82682d8036bc7e7 (127.25.137.194:33035), 527587269ecf4bd2b56c320de373796a (127.25.137.195:34961)
I20260808 02:44:23.205221 1567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.202372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:37801 (local address 127.25.137.194:33035)
0808 02:44:23.202557 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:23.202563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:23.203258 (+ 695us) server_negotiation.cc:407] Connection header received
0808 02:44:23.203455 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:23.203459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:23.203514 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:23.203597 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:23.204566 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:23.204572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:23.204577 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:23.204598 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:23.204630 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:23.204641 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:23.204753 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:23.204844 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 02:44:23.204911 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:44:23.205484 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.202431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:33035 (local address 127.25.137.193:37801)
0808 02:44:23.203134 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.203150 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:23.203316 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:23.203647 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:23.203652 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:23.203676 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:23.204350 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:44:23.204353 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:23.204372 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:44:23.204424 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:23.204727 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:23.204732 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:23.204737 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:44:23.204870 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 02:44:23.204922 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":746,"threads_started":1}
I20260808 02:44:23.205588 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.202372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:60831 (local address 127.25.137.195:34961)
0808 02:44:23.202557 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:23.202563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:23.203645 (+ 1082us) server_negotiation.cc:407] Connection header received
0808 02:44:23.203748 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:23.203753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:23.203848 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:23.203941 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:23.204795 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:23.204800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:23.204803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:23.204821 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:23.204855 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:23.204867 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:23.205000 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:23.205196 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 02:44:23.205272 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:44:23.205885 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.202439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:34961 (local address 127.25.137.193:60831)
0808 02:44:23.203520 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.203538 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:23.203646 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:23.203992 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:23.203996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:23.204018 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:23.204555 (+ 537us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:44:23.204558 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:23.204573 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:44:23.204615 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:23.204979 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:23.204984 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:23.204989 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:44:23.205249 (+ 260us) client_negotiation.cc:240] Negotiation successful
0808 02:44:23.205304 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":30,"thread_start_us":336,"threads_started":1}
I20260808 02:44:23.206765 1271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8e9b260486461ca9b845ed089bdca9" candidate_uuid: "35af09257a104185a0ed3682ff9a66ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9adb4528e649fdb82682d8036bc7e7" is_pre_election: true
I20260808 02:44:23.206804 1469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8e9b260486461ca9b845ed089bdca9" candidate_uuid: "35af09257a104185a0ed3682ff9a66ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "527587269ecf4bd2b56c320de373796a" is_pre_election: true
I20260808 02:44:23.207347 1469 raft_consensus.cc:2454] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35af09257a104185a0ed3682ff9a66ce in term 0.
I20260808 02:44:23.207353 1271 raft_consensus.cc:2454] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35af09257a104185a0ed3682ff9a66ce in term 0.
I20260808 02:44:23.208050 969 leader_election.cc:302] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35af09257a104185a0ed3682ff9a66ce, 527587269ecf4bd2b56c320de373796a; no voters:
I20260808 02:44:23.208412 1581 raft_consensus.cc:2781] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:44:23.208563 1581 raft_consensus.cc:482] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:44:23.208663 1581 raft_consensus.cc:3037] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:23.210897 1581 raft_consensus.cc:504] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35af09257a104185a0ed3682ff9a66ce" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 40221 } } peers { permanent_uuid: "ce9adb4528e649fdb82682d8036bc7e7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33035 } } peers { permanent_uuid: "527587269ecf4bd2b56c320de373796a" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34961 } }
I20260808 02:44:23.211434 1581 leader_election.cc:288] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [CANDIDATE]: Term 1 election: Requested vote from peers ce9adb4528e649fdb82682d8036bc7e7 (127.25.137.194:33035), 527587269ecf4bd2b56c320de373796a (127.25.137.195:34961)
I20260808 02:44:23.211874 1271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8e9b260486461ca9b845ed089bdca9" candidate_uuid: "35af09257a104185a0ed3682ff9a66ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9adb4528e649fdb82682d8036bc7e7"
I20260808 02:44:23.211946 1469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8e9b260486461ca9b845ed089bdca9" candidate_uuid: "35af09257a104185a0ed3682ff9a66ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "527587269ecf4bd2b56c320de373796a"
I20260808 02:44:23.212039 1271 raft_consensus.cc:3037] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:23.212150 1469 raft_consensus.cc:3037] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:23.214394 1271 raft_consensus.cc:2454] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35af09257a104185a0ed3682ff9a66ce in term 1.
I20260808 02:44:23.214519 1469 raft_consensus.cc:2454] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35af09257a104185a0ed3682ff9a66ce in term 1.
I20260808 02:44:23.214823 968 leader_election.cc:302] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35af09257a104185a0ed3682ff9a66ce, ce9adb4528e649fdb82682d8036bc7e7; no voters:
I20260808 02:44:23.215143 1581 raft_consensus.cc:2781] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:23.215544 1581 raft_consensus.cc:686] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 1 LEADER]: Becoming Leader. State: Replica: 35af09257a104185a0ed3682ff9a66ce, State: Running, Role: LEADER
I20260808 02:44:23.215992 1581 consensus_queue.cc:237] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35af09257a104185a0ed3682ff9a66ce" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 40221 } } peers { permanent_uuid: "ce9adb4528e649fdb82682d8036bc7e7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33035 } } peers { permanent_uuid: "527587269ecf4bd2b56c320de373796a" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34961 } }
I20260808 02:44:23.220506 858 catalog_manager.cc:5495] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 35af09257a104185a0ed3682ff9a66ce (127.25.137.193). New cstate: current_term: 1 leader_uuid: "35af09257a104185a0ed3682ff9a66ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35af09257a104185a0ed3682ff9a66ce" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 40221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce9adb4528e649fdb82682d8036bc7e7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "527587269ecf4bd2b56c320de373796a" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34961 } health_report { overall_health: UNKNOWN } } }
I20260808 02:44:23.234428 1562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.232363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49304 (local address 127.25.137.193:40221)
0808 02:44:23.232551 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:23.232558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:23.232619 (+ 61us) server_negotiation.cc:407] Connection header received
0808 02:44:23.232846 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:23.232849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:23.232930 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:23.233033 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:23.233837 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:23.233840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:23.233843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:23.233857 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:23.233889 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:23.233900 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:23.233998 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:23.234102 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 02:44:23.234162 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 02:44:23.238821 1542 tablet.cc:2369] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:44:23.242681 1469 raft_consensus.cc:1261] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a [term 1 FOLLOWER]: Refusing update from remote peer 35af09257a104185a0ed3682ff9a66ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:23.242683 1271 raft_consensus.cc:1261] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7 [term 1 FOLLOWER]: Refusing update from remote peer 35af09257a104185a0ed3682ff9a66ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:23.243561 1581 consensus_queue.cc:1035] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "527587269ecf4bd2b56c320de373796a" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:23.243882 1590 consensus_queue.cc:1035] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce9adb4528e649fdb82682d8036bc7e7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:23.254182 1595 mvcc.cc:205] Tried to move back new op lower bound from 7316099331031883776 to 7316099330936647680. Current Snapshot: MvccSnapshot[applied={T|T < 7316099331031883776}]
I20260808 02:44:23.269630 858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:23.269907 858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:23.273584 858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b59cf1d27bdf4812aaf7243b8a77a503: Sending DeleteTablet for 3 replicas of tablet 2f8e9b260486461ca9b845ed089bdca9
I20260808 02:44:23.274778 1442 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f8e9b260486461ca9b845ed089bdca9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:23 UTC) from {username='slave'} at 127.0.0.1:40294
I20260808 02:44:23.274849 1239 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f8e9b260486461ca9b845ed089bdca9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:23 UTC) from {username='slave'} at 127.0.0.1:55928
I20260808 02:44:23.275858 1611 tablet_replica.cc:330] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7: stopping tablet replica
I20260808 02:44:23.275825 1030 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f8e9b260486461ca9b845ed089bdca9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:23 UTC) from {username='slave'} at 127.0.0.1:49298
I20260808 02:44:23.276422 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 946
I20260808 02:44:23.276437 1611 raft_consensus.cc:2229] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:23.276984 1611 raft_consensus.cc:2258] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:23.277777 1609 tablet_replica.cc:330] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: stopping tablet replica
I20260808 02:44:23.278172 1609 raft_consensus.cc:2229] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:23.278633 1609 raft_consensus.cc:2258] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:23.279417 1612 tablet_replica.cc:330] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce: stopping tablet replica
I20260808 02:44:23.280575 1611 ts_tablet_manager.cc:1905] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:23.281109 1609 ts_tablet_manager.cc:1905] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:23.285148 1612 raft_consensus.cc:2229] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:44:23.285846 1612 raft_consensus.cc:2258] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:23.287925 1611 ts_tablet_manager.cc:1918] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:23.288167 1611 log.cc:1196] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/2f8e9b260486461ca9b845ed089bdca9
I20260808 02:44:23.288152 1612 ts_tablet_manager.cc:1905] T 2f8e9b260486461ca9b845ed089bdca9 P 35af09257a104185a0ed3682ff9a66ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:23.288879 1609 ts_tablet_manager.cc:1918] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:23.289084 1609 log.cc:1196] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/2f8e9b260486461ca9b845ed089bdca9
I20260808 02:44:23.289642 1609 ts_tablet_manager.cc:1939] T 2f8e9b260486461ca9b845ed089bdca9 P 527587269ecf4bd2b56c320de373796a: Deleting consensus metadata
I20260808 02:44:23.288776 1611 ts_tablet_manager.cc:1939] T 2f8e9b260486461ca9b845ed089bdca9 P ce9adb4528e649fdb82682d8036bc7e7: Deleting consensus metadata
I20260808 02:44:23.291553 832 catalog_manager.cc:4841] TS 527587269ecf4bd2b56c320de373796a (127.25.137.195:34961): tablet 2f8e9b260486461ca9b845ed089bdca9 (table test-table [id=c371efa89ce5493eb40b6b0f1afca941]) successfully deleted
I20260808 02:44:23.291572 828 catalog_manager.cc:4841] TS ce9adb4528e649fdb82682d8036bc7e7 (127.25.137.194:33035): tablet 2f8e9b260486461ca9b845ed089bdca9 (table test-table [id=c371efa89ce5493eb40b6b0f1afca941]) successfully deleted
I20260808 02:44:23.294258 1562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.277942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.25.137.193:40221)
0808 02:44:23.278125 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:23.278131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:23.292068 (+ 13937us) server_negotiation.cc:407] Connection header received
0808 02:44:23.292389 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:23.292395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:23.292460 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:23.292553 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:23.293507 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:23.293511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:23.293515 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:23.293530 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:23.293560 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:23.293573 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:23.293686 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:23.293830 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 02:44:23.293900 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:44:23.294947 1120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:23.388881 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 946
I20260808 02:44:23.410394 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 1130
W20260808 02:44:23.410385 828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.137.193:40221 (error 108)
I20260808 02:44:23.412477 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.411970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:40221 (local address 127.0.0.1:49326)
0808 02:44:23.412137 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.412281 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:40221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:44:23.412737 828 catalog_manager.cc:4584] TS 35af09257a104185a0ed3682ff9a66ce (127.25.137.193:40221): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2f8e9b260486461ca9b845ed089bdca9: Network error: Client connection negotiation failed: client connection to 127.25.137.193:40221: connect: Connection refused (error 111)
I20260808 02:44:23.413547 1567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.411200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55942 (local address 127.25.137.194:33035)
0808 02:44:23.411423 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:23.411430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:23.411549 (+ 119us) server_negotiation.cc:407] Connection header received
0808 02:44:23.411755 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:23.411761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:23.411876 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:23.412012 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:23.412896 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:23.412902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:23.412907 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:23.412926 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:23.412987 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:23.413031 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:23.413160 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:23.413242 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 02:44:23.413288 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 02:44:23.414184 1333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:23.466398 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.465916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:40221 (local address 127.0.0.1:49338)
0808 02:44:23.466145 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.466251 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:40221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 02:44:23.467474 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.467069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:40221 (local address 127.0.0.1:49350)
0808 02:44:23.467243 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.467325 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:40221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 02:44:23.504657 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 1130
I20260808 02:44:23.527180 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 1347
I20260808 02:44:23.527863 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.527368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:40221 (local address 127.0.0.1:49358)
0808 02:44:23.527592 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.527683 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:40221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 02:44:23.528971 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.528580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:40221 (local address 127.0.0.1:49362)
0808 02:44:23.528742 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.528819 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:40221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 02:44:23.529836 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.527958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40308 (local address 127.25.137.195:34961)
0808 02:44:23.528151 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:23.528157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:23.528179 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:44:23.528248 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:23.528252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:23.528330 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:23.528447 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:23.529284 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:23.529288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:23.529291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:23.529306 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:23.529338 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:23.529350 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:23.529462 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:23.529500 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 02:44:23.529543 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 02:44:23.530442 1530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:23.609735 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.609231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:40221 (local address 127.0.0.1:49376)
0808 02:44:23.609440 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.609561 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:40221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 02:44:23.610841 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:23.610426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:40221 (local address 127.0.0.1:49386)
0808 02:44:23.610618 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:23.610692 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:40221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260808 02:44:23.626853 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 1347
I20260808 02:44:23.652550 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 803
I20260808 02:44:23.653880 903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:23.762127 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 803
2026-08-08T02:44:23Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2645 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 02:44:23.811839 26151 mini_oidc.cc:152] Starting JWKS server
I20260808 02:44:23.814774 26151 webserver.cc:460] Webserver started at https://127.0.0.1:45889/ using document root <none> and password file <none>
I20260808 02:44:23.815096 26151 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:44:23.815752 26151 webserver.cc:460] Webserver started at http://127.0.0.1:40073/ using document root <none> and password file <none>
2026-08-08T02:44:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:23Z Disabled control of system clock
I20260808 02:44:23.848258 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:39427
--webserver_interface=127.25.137.254
--webserver_port=0
--builtin_ntp_servers=127.25.137.212:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:39427
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45889/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/testchainca.cert with env {}
W20260808 02:44:24.173427 1628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:24.173725 1628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:24.173825 1628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:24.180969 1628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:24.181097 1628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:24.181143 1628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:24.181198 1628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:24.181242 1628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:44:24.181303 1628 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:44:24.181340 1628 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45889/jwks.json
I20260808 02:44:24.191422 1628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:39427
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:39427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45889/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:24.192696 1628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:24.194007 1628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:24.206028 1635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:24.206066 1634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:24.206107 1637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:24.206916 1628 server_base.cc:1034] running on GCE node
I20260808 02:44:24.207690 1628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:24.209098 1628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:24.210312 1628 hybrid_clock.cc:648] HybridClock initialized: now 1786157064210251 us; error 43 us; skew 500 ppm
I20260808 02:44:24.210649 1628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:44:24.212852 1628 webserver.cc:460] Webserver started at http://127.25.137.254:35389/ using document root <none> and password file <none>
I20260808 02:44:24.213550 1628 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:24.213649 1628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:24.213956 1628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:24.216878 1628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "a90e937c8b6646c3b5aeb321b678c023"
format_stamp: "Formatted at 2026-08-08 02:44:24 on dist-test-slave-k5rr"
server_key: "8ffb50fb3968ea7ac6e25a2ff2d58f85"
server_key_iv: "9db6dd9dfd25c98c76d3dfb143cfe13e"
server_key_version: "encryptionkey@0"
I20260808 02:44:24.217711 1628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "a90e937c8b6646c3b5aeb321b678c023"
format_stamp: "Formatted at 2026-08-08 02:44:24 on dist-test-slave-k5rr"
server_key: "8ffb50fb3968ea7ac6e25a2ff2d58f85"
server_key_iv: "9db6dd9dfd25c98c76d3dfb143cfe13e"
server_key_version: "encryptionkey@0"
I20260808 02:44:24.222611 1628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:44:24.226027 1646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:24.227362 1628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:44:24.227525 1628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "a90e937c8b6646c3b5aeb321b678c023"
format_stamp: "Formatted at 2026-08-08 02:44:24 on dist-test-slave-k5rr"
server_key: "8ffb50fb3968ea7ac6e25a2ff2d58f85"
server_key_iv: "9db6dd9dfd25c98c76d3dfb143cfe13e"
server_key_version: "encryptionkey@0"
I20260808 02:44:24.227726 1628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:24.262688 1628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:24.263603 1628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:24.263928 1628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:24.282136 1628 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:39427
I20260808 02:44:24.282168 1722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:39427 every 8 connection(s)
I20260808 02:44:24.283440 1628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:24.287442 1726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:24.288280 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 1628
I20260808 02:44:24.288455 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:24.289124 26151 external_mini_cluster.cc:1428] Setting key a5d17ad11342c050ecc87005d8ffa5af
I20260808 02:44:24.301863 1726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023: Bootstrap starting.
I20260808 02:44:24.304271 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:24.291074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59328 (local address 127.25.137.254:39427)
0808 02:44:24.292213 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:24.292226 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:24.292283 (+ 57us) server_negotiation.cc:407] Connection header received
0808 02:44:24.294079 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:24.294107 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:24.294705 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:24.295149 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:24.295676 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:24.297535 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:24.299026 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:24.300046 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:24.300210 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:24.301508 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:24.301518 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:24.301535 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:24.302221 (+ 686us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:24.302325 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:24.302363 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:24.302472 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:24.302702 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 02:44:24.303016 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":350,"threads_started":1}
I20260808 02:44:24.305548 1726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:24.306564 1726 log.cc:824] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:24.308347 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:44:24.309798 1726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023: No bootstrap required, opened a new log
I20260808 02:44:24.317296 1726 raft_consensus.cc:348] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90e937c8b6646c3b5aeb321b678c023" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39427 } }
I20260808 02:44:24.317625 1726 raft_consensus.cc:374] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:24.317708 1726 raft_consensus.cc:729] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a90e937c8b6646c3b5aeb321b678c023, State: Initialized, Role: FOLLOWER
I20260808 02:44:24.318260 1726 consensus_queue.cc:260] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90e937c8b6646c3b5aeb321b678c023" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39427 } }
I20260808 02:44:24.318480 1726 raft_consensus.cc:388] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:24.318610 1726 raft_consensus.cc:482] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:24.318773 1726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:24.320858 1726 raft_consensus.cc:504] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90e937c8b6646c3b5aeb321b678c023" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39427 } }
I20260808 02:44:24.321364 1726 leader_election.cc:302] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a90e937c8b6646c3b5aeb321b678c023; no voters:
I20260808 02:44:24.321941 1726 leader_election.cc:288] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:24.322069 1732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:24.322722 1732 raft_consensus.cc:686] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [term 1 LEADER]: Becoming Leader. State: Replica: a90e937c8b6646c3b5aeb321b678c023, State: Running, Role: LEADER
I20260808 02:44:24.323808 1732 consensus_queue.cc:237] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90e937c8b6646c3b5aeb321b678c023" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39427 } }
I20260808 02:44:24.323925 1726 sys_catalog.cc:564] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:24.326814 1734 sys_catalog.cc:455] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a90e937c8b6646c3b5aeb321b678c023" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90e937c8b6646c3b5aeb321b678c023" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39427 } } }
I20260808 02:44:24.327054 1736 sys_catalog.cc:455] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a90e937c8b6646c3b5aeb321b678c023. Latest consensus state: current_term: 1 leader_uuid: "a90e937c8b6646c3b5aeb321b678c023" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a90e937c8b6646c3b5aeb321b678c023" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39427 } } }
I20260808 02:44:24.330822 1736 sys_catalog.cc:458] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:24.330819 1734 sys_catalog.cc:458] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:24.331947 1746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:24.337721 1746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:24.346113 1746 catalog_manager.cc:1347] Generated new cluster ID: 9290f1b5fca647bc8874b877c8473aed
I20260808 02:44:24.346235 1746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:24.374204 1746 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:24.376067 1746 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:24.384305 1746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a90e937c8b6646c3b5aeb321b678c023: Generated new TSK 0
I20260808 02:44:24.385358 1746 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 02:44:24.435817 1769 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 02:44:24.436025 1769 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 02:44:24.437366 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:24.414450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59332 (local address 127.25.137.254:39427)
0808 02:44:24.414662 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:24.414669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:24.415083 (+ 414us) server_negotiation.cc:407] Connection header received
0808 02:44:24.415426 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:24.415433 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:24.415538 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:24.415710 (+ 172us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:44:24.416286 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:24.417485 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:24.418973 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:24.419686 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:24.419821 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:24.420432 (+ 611us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:44:24.436580 (+ 16148us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45889/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 02:44:24.436995 (+ 415us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59332: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45889/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 02:44:24.437510 1729 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59332: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45889/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 02:44:24.438920 1765 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:24.414309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:39427 (local address 127.0.0.1:59332)
0808 02:44:24.414975 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:24.414995 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:24.415146 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:24.415735 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:24.415744 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:24.415755 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:44:24.416119 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:24.416129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:24.417640 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:24.417645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:24.418823 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:24.418836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:24.419922 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:24.419927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:24.420064 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:24.420126 (+ 62us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:44:24.436907 (+ 16781us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45889/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 02:44:24.437226 (+ 319us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.137.254:39427: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45889/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":349,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.25.137.254:39427: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45889/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 02:44:24.440742 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 1628
I20260808 02:44:24.441931 1715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:24.540907 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 1628
2026-08-08T02:44:24Z chronyd exiting
I20260808 02:44:24.598995 26151 test_util.cc:175] -----------------------------------------------
I20260808 02:44:24.599123 26151 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157019169307-26151-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (792 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 02:44:24.603116 26151 mini_oidc.cc:152] Starting JWKS server
I20260808 02:44:24.604959 26151 webserver.cc:460] Webserver started at https://127.0.0.1:46725/ using document root <none> and password file <none>
I20260808 02:44:24.605219 26151 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:44:24.605837 26151 webserver.cc:460] Webserver started at http://127.0.0.1:38505/ using document root <none> and password file <none>
2026-08-08T02:44:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:24Z Disabled control of system clock
I20260808 02:44:24.639705 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45717
--webserver_interface=127.25.137.254
--webserver_port=0
--builtin_ntp_servers=127.25.137.212:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:45717
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46725/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/testchainca.cert with env {}
W20260808 02:44:24.956507 1780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:24.956794 1780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:24.956911 1780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:24.964156 1780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:24.964257 1780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:24.964301 1780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:24.964382 1780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:24.964435 1780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:44:24.964519 1780 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:44:24.964566 1780 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46725/jwks.json
I20260808 02:44:24.974457 1780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:45717
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46725/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:24.975731 1780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:24.977034 1780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:24.988554 1785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:24.988511 1786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:24.988586 1794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:24.989560 1780 server_base.cc:1034] running on GCE node
I20260808 02:44:24.990315 1780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:24.991751 1780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:24.992940 1780 hybrid_clock.cc:648] HybridClock initialized: now 1786157064992883 us; error 41 us; skew 500 ppm
I20260808 02:44:24.993283 1780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:44:25.000097 1780 webserver.cc:460] Webserver started at http://127.25.137.254:33719/ using document root <none> and password file <none>
I20260808 02:44:25.000787 1780 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:25.000885 1780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:25.001256 1780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:25.004223 1780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "25cc0a1c318149df975281d979bea49e"
format_stamp: "Formatted at 2026-08-08 02:44:24 on dist-test-slave-k5rr"
server_key: "274312587a3dcd7ea694b9ea83958b23"
server_key_iv: "695fe2184ad7b61c535f75c27ffdd085"
server_key_version: "encryptionkey@0"
I20260808 02:44:25.005052 1780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "25cc0a1c318149df975281d979bea49e"
format_stamp: "Formatted at 2026-08-08 02:44:24 on dist-test-slave-k5rr"
server_key: "274312587a3dcd7ea694b9ea83958b23"
server_key_iv: "695fe2184ad7b61c535f75c27ffdd085"
server_key_version: "encryptionkey@0"
I20260808 02:44:25.010421 1780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:44:25.013926 1802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:25.015283 1780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:44:25.015450 1780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "25cc0a1c318149df975281d979bea49e"
format_stamp: "Formatted at 2026-08-08 02:44:24 on dist-test-slave-k5rr"
server_key: "274312587a3dcd7ea694b9ea83958b23"
server_key_iv: "695fe2184ad7b61c535f75c27ffdd085"
server_key_version: "encryptionkey@0"
I20260808 02:44:25.015661 1780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:25.027019 1780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:25.027776 1780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:25.028062 1780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:25.045595 1780 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:45717
I20260808 02:44:25.045614 1870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:45717 every 8 connection(s)
I20260808 02:44:25.046962 1780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:25.048998 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 1780
I20260808 02:44:25.049217 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:25.049866 26151 external_mini_cluster.cc:1428] Setting key 0d6938725017e7548cbe93c0a9bfa109
I20260808 02:44:25.050921 1871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:25.062678 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:25.051560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40194 (local address 127.25.137.254:45717)
0808 02:44:25.052602 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:25.052616 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:25.052782 (+ 166us) server_negotiation.cc:407] Connection header received
0808 02:44:25.054455 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:25.054476 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:25.054908 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:25.055289 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:25.055842 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.057235 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.058523 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.059170 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.059330 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:25.060474 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:25.060482 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:25.060494 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:25.061076 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:25.061150 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:25.061174 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:25.061299 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:25.061483 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 02:44:25.061770 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":338,"threads_started":1}
I20260808 02:44:25.063333 1871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e: Bootstrap starting.
I20260808 02:44:25.066200 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:44:25.066568 1871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:25.067512 1871 log.cc:824] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:25.070336 1871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e: No bootstrap required, opened a new log
I20260808 02:44:25.077440 1871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cc0a1c318149df975281d979bea49e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45717 } }
I20260808 02:44:25.077788 1871 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:25.077903 1871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25cc0a1c318149df975281d979bea49e, State: Initialized, Role: FOLLOWER
I20260808 02:44:25.078491 1871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cc0a1c318149df975281d979bea49e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45717 } }
I20260808 02:44:25.078706 1871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:25.078831 1871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:25.078981 1871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:25.081050 1871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cc0a1c318149df975281d979bea49e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45717 } }
I20260808 02:44:25.081528 1871 leader_election.cc:302] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25cc0a1c318149df975281d979bea49e; no voters:
I20260808 02:44:25.082141 1871 leader_election.cc:288] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:25.082232 1877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:25.082973 1877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [term 1 LEADER]: Becoming Leader. State: Replica: 25cc0a1c318149df975281d979bea49e, State: Running, Role: LEADER
I20260808 02:44:25.083467 1877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cc0a1c318149df975281d979bea49e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45717 } }
I20260808 02:44:25.084165 1871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:25.086323 1878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25cc0a1c318149df975281d979bea49e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cc0a1c318149df975281d979bea49e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45717 } } }
I20260808 02:44:25.086689 1881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25cc0a1c318149df975281d979bea49e. Latest consensus state: current_term: 1 leader_uuid: "25cc0a1c318149df975281d979bea49e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25cc0a1c318149df975281d979bea49e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45717 } } }
I20260808 02:44:25.087020 1878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:25.087054 1881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:25.088701 1891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:25.093248 1891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:25.100890 1891 catalog_manager.cc:1347] Generated new cluster ID: a91258af5eda459b9cec8ee1059adef0
I20260808 02:44:25.100997 1891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:25.113093 1891 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:25.114370 1891 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:25.125845 1891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25cc0a1c318149df975281d979bea49e: Generated new TSK 0
I20260808 02:44:25.126628 1891 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 02:44:25.161491 1915 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 02:44:25.161620 1915 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 02:44:25.162650 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:25.142876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.25.137.254:45717)
0808 02:44:25.143085 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:25.143092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:25.143585 (+ 493us) server_negotiation.cc:407] Connection header received
0808 02:44:25.143815 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:25.143819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:25.143906 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:25.144054 (+ 148us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:44:25.144641 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.145393 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.147388 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.147857 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.147954 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:25.148518 (+ 564us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:44:25.162092 (+ 13574us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 02:44:25.162412 (+ 320us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40204: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 02:44:25.162768 1875 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40204: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 02:44:25.163018 1914 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:25.142849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45717 (local address 127.0.0.1:40204)
0808 02:44:25.143454 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:25.143475 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:25.143629 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:25.144081 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:25.144087 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:25.144100 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:44:25.144424 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:25.144454 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.145535 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.145541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:25.147225 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:25.147241 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.148030 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.148037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:25.148235 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:25.148281 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:44:25.162336 (+ 14055us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 02:44:25.162622 (+ 286us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.137.254:45717: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":294,"threads_started":1}
I20260808 02:44:25.164206 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 1780
I20260808 02:44:25.165422 1865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:25.258667 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 1780
2026-08-08T02:44:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (716 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 02:44:25.319720 26151 mini_oidc.cc:152] Starting JWKS server
I20260808 02:44:25.321679 26151 webserver.cc:460] Webserver started at https://127.0.0.1:39639/ using document root <none> and password file <none>
I20260808 02:44:25.321902 26151 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:44:25.322548 26151 webserver.cc:460] Webserver started at http://127.0.0.1:45073/ using document root <none> and password file <none>
2026-08-08T02:44:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:25Z Disabled control of system clock
I20260808 02:44:25.357224 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:46493
--webserver_interface=127.25.137.254
--webserver_port=0
--builtin_ntp_servers=127.25.137.212:41081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:46493
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39639/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 02:44:25.689752 1927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:25.690096 1927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:25.690300 1927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:25.697863 1927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:25.698004 1927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:25.698081 1927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:25.698174 1927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:25.698248 1927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:44:25.698346 1927 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:44:25.698407 1927 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39639/jwks.json
I20260808 02:44:25.708144 1927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:46493
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:46493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39639/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:25.709529 1927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:25.711393 1927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:25.723328 1942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:25.723357 1939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:25.723587 1937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:25.724161 1927 server_base.cc:1034] running on GCE node
I20260808 02:44:25.724901 1927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:25.726286 1927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:25.727489 1927 hybrid_clock.cc:648] HybridClock initialized: now 1786157065727427 us; error 57 us; skew 500 ppm
I20260808 02:44:25.727819 1927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:44:25.730016 1927 webserver.cc:460] Webserver started at http://127.25.137.254:45699/ using document root <none> and password file <none>
I20260808 02:44:25.730664 1927 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:25.730749 1927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:25.731055 1927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:25.733829 1927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "4e789b21fb6a4681a4fa5c9a0e801de1"
format_stamp: "Formatted at 2026-08-08 02:44:25 on dist-test-slave-k5rr"
server_key: "70216e04fe1dcf4669c482b418d2cbd0"
server_key_iv: "9b93402fe4076c4f03ded2efcaddece7"
server_key_version: "encryptionkey@0"
I20260808 02:44:25.734583 1927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "4e789b21fb6a4681a4fa5c9a0e801de1"
format_stamp: "Formatted at 2026-08-08 02:44:25 on dist-test-slave-k5rr"
server_key: "70216e04fe1dcf4669c482b418d2cbd0"
server_key_iv: "9b93402fe4076c4f03ded2efcaddece7"
server_key_version: "encryptionkey@0"
I20260808 02:44:25.739401 1927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:44:25.742655 1949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:25.744040 1927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:44:25.744221 1927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "4e789b21fb6a4681a4fa5c9a0e801de1"
format_stamp: "Formatted at 2026-08-08 02:44:25 on dist-test-slave-k5rr"
server_key: "70216e04fe1dcf4669c482b418d2cbd0"
server_key_iv: "9b93402fe4076c4f03ded2efcaddece7"
server_key_version: "encryptionkey@0"
I20260808 02:44:25.744436 1927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:25.780923 1927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:25.781850 1927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:25.782158 1927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:25.800145 1927 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:46493
I20260808 02:44:25.800192 2024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:46493 every 8 connection(s)
I20260808 02:44:25.801519 1927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:25.805378 2025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:25.808099 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 1927
I20260808 02:44:25.808279 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:25.808912 26151 external_mini_cluster.cc:1428] Setting key 5a0b442ed437e56c43eea89e32f8e1fa
I20260808 02:44:25.817788 2025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1: Bootstrap starting.
I20260808 02:44:25.821679 2025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:25.822799 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:25.810842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.25.137.254:46493)
0808 02:44:25.811863 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:25.811874 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:25.811915 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:44:25.813637 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:25.813660 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:25.814120 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:25.814456 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:25.815088 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.816405 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.818331 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.819003 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.819132 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:25.820453 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:25.820460 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:25.820473 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:25.820988 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:25.821103 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:25.821130 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:25.821216 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:25.821503 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 02:44:25.821820 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":348,"threads_started":1}
I20260808 02:44:25.822839 2025 log.cc:824] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:25.826105 2025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1: No bootstrap required, opened a new log
I20260808 02:44:25.826669 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:44:25.833513 2025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e789b21fb6a4681a4fa5c9a0e801de1" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46493 } }
I20260808 02:44:25.833913 2025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:25.834031 2025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e789b21fb6a4681a4fa5c9a0e801de1, State: Initialized, Role: FOLLOWER
I20260808 02:44:25.834529 2025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e789b21fb6a4681a4fa5c9a0e801de1" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46493 } }
I20260808 02:44:25.834743 2025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:25.834843 2025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:25.834965 2025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:25.836983 2025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e789b21fb6a4681a4fa5c9a0e801de1" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46493 } }
I20260808 02:44:25.837474 2025 leader_election.cc:302] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e789b21fb6a4681a4fa5c9a0e801de1; no voters:
I20260808 02:44:25.837972 2025 leader_election.cc:288] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:25.838099 2031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:25.839159 2031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [term 1 LEADER]: Becoming Leader. State: Replica: 4e789b21fb6a4681a4fa5c9a0e801de1, State: Running, Role: LEADER
I20260808 02:44:25.839655 2031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e789b21fb6a4681a4fa5c9a0e801de1" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46493 } }
I20260808 02:44:25.840019 2025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:25.842484 2033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e789b21fb6a4681a4fa5c9a0e801de1. Latest consensus state: current_term: 1 leader_uuid: "4e789b21fb6a4681a4fa5c9a0e801de1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e789b21fb6a4681a4fa5c9a0e801de1" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46493 } } }
I20260808 02:44:25.842458 2032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e789b21fb6a4681a4fa5c9a0e801de1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e789b21fb6a4681a4fa5c9a0e801de1" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46493 } } }
I20260808 02:44:25.843005 2033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:25.843060 2032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:25.843909 2049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:25.849362 2049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:25.858162 2049 catalog_manager.cc:1347] Generated new cluster ID: 27a2b251ecca4c2d882dabd073ee4e1d
I20260808 02:44:25.858278 2049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:25.890545 2049 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:25.891912 2049 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:25.903395 2049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e789b21fb6a4681a4fa5c9a0e801de1: Generated new TSK 0
I20260808 02:44:25.904286 2049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:44:25.930142 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:25.921497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49610 (local address 127.25.137.254:46493)
0808 02:44:25.921700 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:25.921706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:25.922112 (+ 406us) server_negotiation.cc:407] Connection header received
0808 02:44:25.922337 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:25.922343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:25.922441 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:25.922595 (+ 154us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:44:25.923125 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.924257 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.925797 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.926490 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.926623 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:25.927111 (+ 488us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:44:25.929517 (+ 2406us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39639/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 02:44:25.929846 (+ 329us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49610: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39639/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 02:44:25.930274 2028 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49610: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39639/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 02:44:25.931882 2064 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:25.921352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:46493 (local address 127.0.0.1:49610)
0808 02:44:25.922016 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:25.922033 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:25.922160 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:25.922598 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:25.922601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:25.922611 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:44:25.922973 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:25.922985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.924431 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.924435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:25.925633 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:25.925648 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:25.926668 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:25.926673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:25.926815 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:25.926849 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:44:25.929742 (+ 2893us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39639/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 02:44:25.931462 (+ 1720us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.137.254:46493: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39639/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":1436,"thread_start_us":381,"threads_started":1}
I20260808 02:44:25.933773 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 1927
I20260808 02:44:25.934862 2019 generic_service.cc:196] Checking for leaks (request via RPC)
E20260808 02:44:25.941316 2066 webserver.cc:533] Webserver: sslize failed:
E20260808 02:44:25.941408 2066 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 02:44:26.030589 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 1927
2026-08-08T02:44:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (770 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T02:44:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:26Z Disabled control of system clock
I20260808 02:44:26.124065 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:43185
--webserver_interface=127.25.137.254
--webserver_port=0
--builtin_ntp_servers=127.25.137.212:33331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:43185
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/testchainca.cert with env {}
W20260808 02:44:26.456007 2071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:26.456308 2071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:26.456446 2071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:26.463617 2071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:26.463745 2071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:26.463805 2071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:26.463864 2071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:26.463910 2071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:44:26.463971 2071 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:44:26.464010 2071 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 02:44:26.473668 2071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:43185
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:43185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:26.474906 2071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:26.476122 2071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:26.488626 2080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:26.488891 2083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:26.489023 2079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:26.489449 2071 server_base.cc:1034] running on GCE node
I20260808 02:44:26.490216 2071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:26.491578 2071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:26.492789 2071 hybrid_clock.cc:648] HybridClock initialized: now 1786157066492729 us; error 46 us; skew 500 ppm
I20260808 02:44:26.493201 2071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:44:26.495484 2071 webserver.cc:460] Webserver started at http://127.25.137.254:40869/ using document root <none> and password file <none>
I20260808 02:44:26.496165 2071 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:26.496281 2071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:26.496594 2071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:26.499536 2071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "0ba5aa5d30ac4950ab0fb059425075db"
format_stamp: "Formatted at 2026-08-08 02:44:26 on dist-test-slave-k5rr"
server_key: "2f5b5fdde1bbd9709323bd42f9ab95b5"
server_key_iv: "1d7ec738fef11be6181fbda96d297852"
server_key_version: "encryptionkey@0"
I20260808 02:44:26.500378 2071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "0ba5aa5d30ac4950ab0fb059425075db"
format_stamp: "Formatted at 2026-08-08 02:44:26 on dist-test-slave-k5rr"
server_key: "2f5b5fdde1bbd9709323bd42f9ab95b5"
server_key_iv: "1d7ec738fef11be6181fbda96d297852"
server_key_version: "encryptionkey@0"
I20260808 02:44:26.505525 2071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:44:26.508975 2092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:26.510367 2071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:26.510545 2071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "0ba5aa5d30ac4950ab0fb059425075db"
format_stamp: "Formatted at 2026-08-08 02:44:26 on dist-test-slave-k5rr"
server_key: "2f5b5fdde1bbd9709323bd42f9ab95b5"
server_key_iv: "1d7ec738fef11be6181fbda96d297852"
server_key_version: "encryptionkey@0"
I20260808 02:44:26.510746 2071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:26.530938 2071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:26.531811 2071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:26.532099 2071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:26.549840 2071 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:43185
I20260808 02:44:26.549889 2175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:43185 every 8 connection(s)
I20260808 02:44:26.551120 2071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:26.554966 2176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:26.555589 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 2071
I20260808 02:44:26.555831 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:26.556851 26151 external_mini_cluster.cc:1428] Setting key 057175f7cb91f35ab9099768d381bf9f
I20260808 02:44:26.567849 2176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db: Bootstrap starting.
I20260808 02:44:26.570096 2181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:26.558862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53062 (local address 127.25.137.254:43185)
0808 02:44:26.559896 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:26.559906 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:26.559953 (+ 47us) server_negotiation.cc:407] Connection header received
0808 02:44:26.561914 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:26.561937 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:26.562394 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:26.562757 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:26.563313 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:26.564561 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:26.565888 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:26.566559 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:26.566737 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:26.567848 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:26.567855 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:26.567868 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:44:26.568432 (+ 564us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:26.568508 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:44:26.568529 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:26.568628 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:26.568859 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 02:44:26.569179 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":334,"threads_started":1}
I20260808 02:44:26.571249 2176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:26.572382 2176 log.cc:824] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:26.573829 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:44:26.575336 2176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db: No bootstrap required, opened a new log
I20260808 02:44:26.583462 2176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ba5aa5d30ac4950ab0fb059425075db" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43185 } }
I20260808 02:44:26.583845 2176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:26.583952 2176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ba5aa5d30ac4950ab0fb059425075db, State: Initialized, Role: FOLLOWER
I20260808 02:44:26.584513 2176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ba5aa5d30ac4950ab0fb059425075db" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43185 } }
I20260808 02:44:26.584740 2176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:26.584883 2176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:26.585085 2176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:26.587347 2176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ba5aa5d30ac4950ab0fb059425075db" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43185 } }
I20260808 02:44:26.587867 2176 leader_election.cc:302] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ba5aa5d30ac4950ab0fb059425075db; no voters:
I20260808 02:44:26.588452 2176 leader_election.cc:288] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:26.588577 2186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:26.589260 2186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [term 1 LEADER]: Becoming Leader. State: Replica: 0ba5aa5d30ac4950ab0fb059425075db, State: Running, Role: LEADER
I20260808 02:44:26.589757 2186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ba5aa5d30ac4950ab0fb059425075db" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43185 } }
I20260808 02:44:26.590813 2176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:26.592455 2187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ba5aa5d30ac4950ab0fb059425075db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ba5aa5d30ac4950ab0fb059425075db" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43185 } } }
I20260808 02:44:26.592564 2188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ba5aa5d30ac4950ab0fb059425075db. Latest consensus state: current_term: 1 leader_uuid: "0ba5aa5d30ac4950ab0fb059425075db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ba5aa5d30ac4950ab0fb059425075db" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43185 } } }
I20260808 02:44:26.592850 2187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:26.592850 2188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:26.595454 2193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:26.601824 2193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:26.610437 2193 catalog_manager.cc:1347] Generated new cluster ID: 37aa8c94bbb3420b9a7006a0bd299d7f
I20260808 02:44:26.610563 2193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:26.630976 2193 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:26.632835 2193 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:26.660154 2193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ba5aa5d30ac4950ab0fb059425075db: Generated new TSK 0
I20260808 02:44:26.661093 2193 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:44:26.724581 2181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:26.679764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53078 (local address 127.25.137.254:43185)
0808 02:44:26.679979 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:26.679985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:26.680339 (+ 354us) server_negotiation.cc:407] Connection header received
0808 02:44:26.680566 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:26.680575 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:26.680646 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:26.680773 (+ 127us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:44:26.681422 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:26.682596 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:26.684668 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:26.685278 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:26.685389 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:26.686026 (+ 637us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:44:26.723955 (+ 37929us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 02:44:26.724324 (+ 369us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53078: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":93}
W20260808 02:44:26.724725 2181 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53078: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 02:44:26.725082 2220 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:26.679589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43185 (local address 127.0.0.1:53078)
0808 02:44:26.680217 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:26.680237 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:26.680399 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:26.680811 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:26.680817 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:26.680831 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:44:26.681152 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:26.681165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:26.682753 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:26.682759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:26.684478 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:26.684495 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:26.685477 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:26.685484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:26.685678 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:26.685728 (+ 50us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:44:26.724305 (+ 38577us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 02:44:26.724643 (+ 338us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.137.254:43185: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":308,"threads_started":1}
I20260808 02:44:26.726470 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 2071
I20260808 02:44:26.727622 2167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:26.826005 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 2071
2026-08-08T02:44:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (775 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (413 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (412 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:44:27 dist-test-slave-k5rr krb5kdc[2241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:44:27 dist-test-slave-k5rr krb5kdc[2241](info): set up 2 sockets
Aug 08 02:44:27 dist-test-slave-k5rr krb5kdc[2241](info): commencing operation
krb5kdc: starting...
W20260808 02:44:28.852665 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:44:29 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:44:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:44:29.092375 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:38879
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:38879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:29.416841 2264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:29.417157 2264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:29.417299 2264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:29.424332 2264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:29.424427 2264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:29.424502 2264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:29.424576 2264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:29.424634 2264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:29.434273 2264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:38879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:38879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:29.435494 2264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:29.436800 2264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:29.447786 2273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:29.447844 2279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:29.447918 2276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:29.448832 2264 server_base.cc:1034] running on GCE node
I20260808 02:44:29.449558 2264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:29.450834 2264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:29.452064 2264 hybrid_clock.cc:648] HybridClock initialized: now 1786157069452002 us; error 54 us; skew 500 ppm
I20260808 02:44:29.452394 2264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:29 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157069, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:29.455356 2264 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:44:29.456772 2264 webserver.cc:460] Webserver started at http://127.25.137.254:35047/ using document root <none> and password file <none>
I20260808 02:44:29.457477 2264 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:29.457587 2264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:29.457896 2264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:29.461117 2264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "7780f5428ced423e9fb9eae4d4348a43"
format_stamp: "Formatted at 2026-08-08 02:44:29 on dist-test-slave-k5rr"
server_key: "f12049ec0a9780420ae63ea3ddec2235"
server_key_iv: "d248d753f2545e9c60402404d45199bc"
server_key_version: "encryptionkey@0"
I20260808 02:44:29.461895 2264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "7780f5428ced423e9fb9eae4d4348a43"
format_stamp: "Formatted at 2026-08-08 02:44:29 on dist-test-slave-k5rr"
server_key: "f12049ec0a9780420ae63ea3ddec2235"
server_key_iv: "d248d753f2545e9c60402404d45199bc"
server_key_version: "encryptionkey@0"
I20260808 02:44:29.466862 2264 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:44:29.470281 2287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:29.471611 2264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:29.471817 2264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "7780f5428ced423e9fb9eae4d4348a43"
format_stamp: "Formatted at 2026-08-08 02:44:29 on dist-test-slave-k5rr"
server_key: "f12049ec0a9780420ae63ea3ddec2235"
server_key_iv: "d248d753f2545e9c60402404d45199bc"
server_key_version: "encryptionkey@0"
I20260808 02:44:29.471999 2264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:29.496511 2264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:29.500569 2264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:29.500844 2264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:29.518292 2264 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:38879
I20260808 02:44:29.518343 2366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:38879 every 8 connection(s)
I20260808 02:44:29.519593 2264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:29.523226 2368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:29.525648 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 2264
I20260808 02:44:29.525894 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:29.526680 26151 external_mini_cluster.cc:1428] Setting key db0a63c620bdaa6820cc1489f7c6081f
I20260808 02:44:29.535028 2368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43: Bootstrap starting.
I20260808 02:44:29.538024 2368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:29.538976 2368 log.cc:824] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:44:29 dist-test-slave-k5rr krb5kdc[2241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:29.541702 2368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43: No bootstrap required, opened a new log
I20260808 02:44:29.548172 2373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:29.528922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.25.137.254:38879)
0808 02:44:29.529717 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:29.529728 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:29.529769 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:44:29.531474 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:29.531509 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:29.531899 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:29.532273 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:29.533811 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:29.535140 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:29.536384 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:29.537085 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:29.537245 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:29.541128 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:29.541136 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:29.541150 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:29.541190 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:29.544156 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:29.544805 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:29.544815 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:29.544821 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:29.545002 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:29.545427 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:29.545430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:29.545433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:29.545989 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:29.546270 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:29.546732 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 02:44:29.547160 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":343,"threads_started":1}
I20260808 02:44:29.549651 2368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7780f5428ced423e9fb9eae4d4348a43" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38879 } }
I20260808 02:44:29.550032 2368 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:29.550124 2368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7780f5428ced423e9fb9eae4d4348a43, State: Initialized, Role: FOLLOWER
I20260808 02:44:29.550666 2368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7780f5428ced423e9fb9eae4d4348a43" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38879 } }
I20260808 02:44:29.550923 2368 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:29.551055 2368 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:29.551245 2368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:29.553547 2368 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7780f5428ced423e9fb9eae4d4348a43" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38879 } }
I20260808 02:44:29.553993 2368 leader_election.cc:302] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7780f5428ced423e9fb9eae4d4348a43; no voters:
I20260808 02:44:29.554553 2368 leader_election.cc:288] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:29.554708 2375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:29.555401 2375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [term 1 LEADER]: Becoming Leader. State: Replica: 7780f5428ced423e9fb9eae4d4348a43, State: Running, Role: LEADER
I20260808 02:44:29.555946 2375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7780f5428ced423e9fb9eae4d4348a43" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38879 } }
I20260808 02:44:29.556422 2368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:29.558674 2377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7780f5428ced423e9fb9eae4d4348a43. Latest consensus state: current_term: 1 leader_uuid: "7780f5428ced423e9fb9eae4d4348a43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7780f5428ced423e9fb9eae4d4348a43" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38879 } } }
I20260808 02:44:29.558712 2376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7780f5428ced423e9fb9eae4d4348a43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7780f5428ced423e9fb9eae4d4348a43" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38879 } } }
I20260808 02:44:29.559140 2377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:29.559187 2376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:29.561138 2382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:29.566699 2382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:29.577109 2382 catalog_manager.cc:1347] Generated new cluster ID: 35feb2d058664c78afb8eee1b1101a06
I20260808 02:44:29.577245 2382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:29.597648 2382 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:29.599704 2382 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:29.614179 2382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7780f5428ced423e9fb9eae4d4348a43: Generated new TSK 0
I20260808 02:44:29.615262 2382 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:44:29.678390 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38879
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:30.004627 2405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:30.004946 2405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:30.005086 2405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:30.012193 2405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:30.012292 2405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:30.012404 2405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:44:30.021874 2405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:30.023034 2405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:30.024264 2405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:30.036734 2416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:30.036835 2414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:30.036873 2413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:30.037815 2405 server_base.cc:1034] running on GCE node
I20260808 02:44:30.038403 2405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:30.039263 2405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:30.040450 2405 hybrid_clock.cc:648] HybridClock initialized: now 1786157070040396 us; error 51 us; skew 500 ppm
I20260808 02:44:30.040731 2405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:30 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157070, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:30.051113 2405 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:44:30.052575 2405 webserver.cc:460] Webserver started at http://127.25.137.193:34093/ using document root <none> and password file <none>
I20260808 02:44:30.053282 2405 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:30.053411 2405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:30.053699 2405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:30.056553 2405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "2cdba85827fa4057ba6a0ed32f491ee6"
format_stamp: "Formatted at 2026-08-08 02:44:30 on dist-test-slave-k5rr"
server_key: "0c4e5952545fe171a5eb7bbca063113a"
server_key_iv: "ccab73af879eca61cc77636ddedcfac7"
server_key_version: "encryptionkey@0"
I20260808 02:44:30.057467 2405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "2cdba85827fa4057ba6a0ed32f491ee6"
format_stamp: "Formatted at 2026-08-08 02:44:30 on dist-test-slave-k5rr"
server_key: "0c4e5952545fe171a5eb7bbca063113a"
server_key_iv: "ccab73af879eca61cc77636ddedcfac7"
server_key_version: "encryptionkey@0"
I20260808 02:44:30.062460 2405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 02:44:30.065970 2427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:30.067436 2405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:44:30.067615 2405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "2cdba85827fa4057ba6a0ed32f491ee6"
format_stamp: "Formatted at 2026-08-08 02:44:30 on dist-test-slave-k5rr"
server_key: "0c4e5952545fe171a5eb7bbca063113a"
server_key_iv: "ccab73af879eca61cc77636ddedcfac7"
server_key_version: "encryptionkey@0"
I20260808 02:44:30.067852 2405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:30.098059 2405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:30.102476 2405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:30.102799 2405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:30.103860 2405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:30.105361 2405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:30.105445 2405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:30.105579 2405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:30.105649 2405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:30.136656 2405 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:33185
I20260808 02:44:30.136693 2596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:33185 every 8 connection(s)
I20260808 02:44:30.138092 2405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:44:30.143183 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 2405
I20260808 02:44:30.143414 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:44:30.144069 26151 external_mini_cluster.cc:1428] Setting key 266473787e75cb5b8fc151968a493b10
Aug 08 02:44:30 dist-test-slave-k5rr krb5kdc[2241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157070, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:30.158046 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:30.140717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:53975 (local address 127.25.137.254:38879)
0808 02:44:30.141192 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:30.141197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:30.142148 (+ 951us) server_negotiation.cc:407] Connection header received
0808 02:44:30.144011 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:30.144018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:30.144104 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:30.144198 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:30.145759 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:30.146509 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:30.148044 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:30.148460 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:30.148576 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:30.152295 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:30.152301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:30.152305 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:30.152339 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:30.154891 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:30.155615 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:30.155619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:30.155622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:30.155744 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:30.156213 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:30.156218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:30.156222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:30.156507 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:30.156645 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:30.157561 (+ 916us) server_negotiation.cc:299] Negotiation successful
0808 02:44:30.157758 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":262,"threads_started":1}
I20260808 02:44:30.159464 2604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:30.141262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:38879 (local address 127.25.137.193:53975)
0808 02:44:30.141983 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:30.142028 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:30.143742 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:30.144482 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:30.144490 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:30.144976 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:30.145576 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:30.145595 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:30.146658 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:30.146662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:30.147895 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:30.147907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:30.148696 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:30.148703 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:30.148933 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:30.149646 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:30.149669 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:30.152061 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:30.155034 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:30.155044 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:30.155061 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:30.155490 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:30.155888 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:30.155894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:30.155898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:30.156095 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:30.156635 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:30.156643 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:30.157092 (+ 449us) client_negotiation.cc:769] Sending connection context
0808 02:44:30.157408 (+ 316us) client_negotiation.cc:240] Negotiation successful
0808 02:44:30.157932 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":301,"threads_started":1}
I20260808 02:44:30.161064 2597 heartbeater.cc:347] Connected to a master server at 127.25.137.254:38879
I20260808 02:44:30.161377 2597 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:30.162062 2597 heartbeater.cc:511] Master 127.25.137.254:38879 requested a full tablet report, sending...
I20260808 02:44:30.164180 2311 ts_manager.cc:194] Registered new tserver with Master: 2cdba85827fa4057ba6a0ed32f491ee6 (127.25.137.193:33185)
I20260808 02:44:30.166009 2311 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:53975
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:44:30.198257 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38879
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:30.523893 2608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:30.524152 2608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:30.524279 2608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:30.531469 2608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:30.531595 2608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:30.531746 2608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:44:30.541474 2608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:30.542694 2608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:30.543955 2608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:30.556720 2616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:30.556689 2618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:30.556643 2624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:30.557822 2608 server_base.cc:1034] running on GCE node
I20260808 02:44:30.558386 2608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:30.559319 2608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:30.560560 2608 hybrid_clock.cc:648] HybridClock initialized: now 1786157070560482 us; error 59 us; skew 500 ppm
I20260808 02:44:30.560886 2608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:30 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157070, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:30.563925 2608 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:44:30.565443 2608 webserver.cc:460] Webserver started at http://127.25.137.194:39305/ using document root <none> and password file <none>
I20260808 02:44:30.566125 2608 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:30.566244 2608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:30.566545 2608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:30.569593 2608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "5785ba970c8b476abf9912b331abf526"
format_stamp: "Formatted at 2026-08-08 02:44:30 on dist-test-slave-k5rr"
server_key: "52af740b9d897b1a5a7c690aa608721d"
server_key_iv: "4e2cb808dc77d96cef7d390148fa19da"
server_key_version: "encryptionkey@0"
I20260808 02:44:30.570374 2608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "5785ba970c8b476abf9912b331abf526"
format_stamp: "Formatted at 2026-08-08 02:44:30 on dist-test-slave-k5rr"
server_key: "52af740b9d897b1a5a7c690aa608721d"
server_key_iv: "4e2cb808dc77d96cef7d390148fa19da"
server_key_version: "encryptionkey@0"
I20260808 02:44:30.575771 2608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:44:30.579285 2638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:30.580669 2608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 02:44:30.580837 2608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "5785ba970c8b476abf9912b331abf526"
format_stamp: "Formatted at 2026-08-08 02:44:30 on dist-test-slave-k5rr"
server_key: "52af740b9d897b1a5a7c690aa608721d"
server_key_iv: "4e2cb808dc77d96cef7d390148fa19da"
server_key_version: "encryptionkey@0"
I20260808 02:44:30.581046 2608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:30.599833 2608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:30.603966 2608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:30.604236 2608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:30.605242 2608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:30.606580 2608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:30.606664 2608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:30.606801 2608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:30.606854 2608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:30.637050 2608 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:46173
I20260808 02:44:30.637104 2811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:46173 every 8 connection(s)
I20260808 02:44:30.638305 2608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:44:30.640975 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 2608
I20260808 02:44:30.641202 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:44:30.641618 26151 external_mini_cluster.cc:1428] Setting key 78855e21b7a35130705643208c225837
Aug 08 02:44:30 dist-test-slave-k5rr krb5kdc[2241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157070, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:30.658789 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:30.640979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:51945 (local address 127.25.137.254:38879)
0808 02:44:30.641202 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:30.641208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:30.642514 (+ 1306us) server_negotiation.cc:407] Connection header received
0808 02:44:30.644417 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:30.644423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:30.644502 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:30.644606 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:30.646370 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:30.647269 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:30.648822 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:30.649429 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:30.649540 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:30.653104 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:30.653111 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:30.653116 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:30.653149 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:30.655625 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:30.656483 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:30.656487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:30.656489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:30.656612 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:30.657072 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:30.657076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:30.657078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:30.657398 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:30.657516 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:30.658356 (+ 840us) server_negotiation.cc:299] Negotiation successful
0808 02:44:30.658543 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 02:44:30.660074 2814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:30.641530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:38879 (local address 127.25.137.194:51945)
0808 02:44:30.642354 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:30.642409 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:30.644156 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:30.645143 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:30.645151 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:30.645646 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:30.646200 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:30.646217 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:30.647431 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:30.647435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:30.648676 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:30.648689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:30.649653 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:30.649658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:30.649895 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:30.650533 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:30.650560 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:30.652839 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:30.655771 (+ 2932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:30.655780 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:30.655798 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:30.656386 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:30.656716 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:30.656720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:30.656723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:30.656952 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:30.657530 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:30.657540 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:30.657953 (+ 413us) client_negotiation.cc:769] Sending connection context
0808 02:44:30.658228 (+ 275us) client_negotiation.cc:240] Negotiation successful
0808 02:44:30.658730 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":337,"threads_started":1}
I20260808 02:44:30.661857 2812 heartbeater.cc:347] Connected to a master server at 127.25.137.254:38879
I20260808 02:44:30.662144 2812 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:30.662853 2812 heartbeater.cc:511] Master 127.25.137.254:38879 requested a full tablet report, sending...
I20260808 02:44:30.664294 2311 ts_manager.cc:194] Registered new tserver with Master: 5785ba970c8b476abf9912b331abf526 (127.25.137.194:46173)
I20260808 02:44:30.665192 2311 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:51945
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:44:30.696127 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38879
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:31.022140 2817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:31.022406 2817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:31.022500 2817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:31.030120 2817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:31.030228 2817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:31.030349 2817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:44:31.039951 2817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:31.041221 2817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:31.042479 2817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:31.054920 2828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:31.054961 2826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:31.054963 2832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:31.056087 2817 server_base.cc:1034] running on GCE node
I20260808 02:44:31.056677 2817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:31.057660 2817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:31.058861 2817 hybrid_clock.cc:648] HybridClock initialized: now 1786157071058819 us; error 34 us; skew 500 ppm
I20260808 02:44:31.059146 2817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:31.062271 2817 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:44:31.063707 2817 webserver.cc:460] Webserver started at http://127.25.137.195:40291/ using document root <none> and password file <none>
I20260808 02:44:31.064363 2817 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:31.064476 2817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:31.064766 2817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:31.067670 2817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "8f64b3b95a8648308a9837d923258098"
format_stamp: "Formatted at 2026-08-08 02:44:31 on dist-test-slave-k5rr"
server_key: "f4e070d30535ab2cdd65ff39cbdaebab"
server_key_iv: "241f948e52a69d46aca3e2951658c800"
server_key_version: "encryptionkey@0"
I20260808 02:44:31.068436 2817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "8f64b3b95a8648308a9837d923258098"
format_stamp: "Formatted at 2026-08-08 02:44:31 on dist-test-slave-k5rr"
server_key: "f4e070d30535ab2cdd65ff39cbdaebab"
server_key_iv: "241f948e52a69d46aca3e2951658c800"
server_key_version: "encryptionkey@0"
I20260808 02:44:31.073550 2817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:44:31.077070 2845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:31.078367 2817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:31.078500 2817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "8f64b3b95a8648308a9837d923258098"
format_stamp: "Formatted at 2026-08-08 02:44:31 on dist-test-slave-k5rr"
server_key: "f4e070d30535ab2cdd65ff39cbdaebab"
server_key_iv: "241f948e52a69d46aca3e2951658c800"
server_key_version: "encryptionkey@0"
I20260808 02:44:31.078673 2817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:31.096493 2817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:31.100701 2817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:31.100976 2817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:31.102006 2817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:31.103384 2817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:31.103464 2817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:31.103590 2817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:31.103645 2817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:31.135079 2817 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:44547
I20260808 02:44:31.135124 3025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:44547 every 8 connection(s)
I20260808 02:44:31.136502 2817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:44:31.140362 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 2817
I20260808 02:44:31.140570 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:44:31.141158 26151 external_mini_cluster.cc:1428] Setting key deca5af92f1f8106f74fd513e1f0c181
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:31.157124 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:31.139148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:36859 (local address 127.25.137.254:38879)
0808 02:44:31.139366 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:31.139373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:31.140643 (+ 1270us) server_negotiation.cc:407] Connection header received
0808 02:44:31.142897 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:31.142902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:31.142973 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:31.143074 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:31.144621 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.145445 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.147063 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.147506 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.147613 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:31.151056 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:31.151061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:31.151065 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:31.151103 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:31.153835 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.154514 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.154520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.154523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.154701 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.155209 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.155213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.155215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.155559 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:31.155681 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:31.156575 (+ 894us) server_negotiation.cc:299] Negotiation successful
0808 02:44:31.156785 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:44:31.158126 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:31.139612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:38879 (local address 127.25.137.195:36859)
0808 02:44:31.140424 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:31.140529 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:31.142637 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:31.143282 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:31.143291 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:31.143811 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:31.144471 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:31.144489 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.145618 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.145622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:31.146906 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:31.146921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.147740 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.147745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:31.147961 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:31.148600 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:31.148627 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:31.150832 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:31.153963 (+ 3131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:31.153970 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:31.153982 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:31.154400 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:31.154853 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:31.154858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:31.154861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:31.155099 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:31.155705 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:31.155714 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:31.156076 (+ 362us) client_negotiation.cc:769] Sending connection context
0808 02:44:31.156476 (+ 400us) client_negotiation.cc:240] Negotiation successful
0808 02:44:31.156958 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":328,"threads_started":1}
I20260808 02:44:31.159401 3027 heartbeater.cc:347] Connected to a master server at 127.25.137.254:38879
I20260808 02:44:31.159677 3027 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:31.160395 3027 heartbeater.cc:511] Master 127.25.137.254:38879 requested a full tablet report, sending...
I20260808 02:44:31.161788 2311 ts_manager.cc:194] Registered new tserver with Master: 8f64b3b95a8648308a9837d923258098 (127.25.137.195:44547)
I20260808 02:44:31.162573 2311 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:36859
I20260808 02:44:31.169047 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:44:31.169171 2597 heartbeater.cc:503] Master 127.25.137.254:38879 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/test-admin.keytab.
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:31.199528 26151 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 02:44:31.204978 3039 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:31.202399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.25.137.193:33185)
0808 02:44:31.203077 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:31.203084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:31.203111 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:44:31.203175 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:31.203182 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:31.203399 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:31.203528 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:31.204548 (+ 1020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59566: BlockingRecv error: recv got EOF from 127.0.0.1:59566 (error 108)
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":356,"threads_started":1}
W20260808 02:44:31.205214 26151 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5cc' of type FILE
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:31.206797 26151 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:31.208062 26151 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:44:31.222926 3039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:31.209692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59582 (local address 127.25.137.193:33185)
0808 02:44:31.209867 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:31.209873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:31.210034 (+ 161us) server_negotiation.cc:407] Connection header received
0808 02:44:31.210208 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:31.210212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:31.210269 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:31.210365 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:31.211358 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.212311 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.213599 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.214143 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.214265 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:31.217800 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:31.217812 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:31.217827 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:31.217859 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:31.220461 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.221181 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.221188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.221190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.221329 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.221616 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.221619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.221621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.221969 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:31.222196 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:31.222456 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 02:44:31.222654 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 02:44:31.228508 3039 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:31.226748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59590 (local address 127.25.137.193:33185)
0808 02:44:31.226919 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:31.226924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:31.227241 (+ 317us) server_negotiation.cc:407] Connection header received
0808 02:44:31.227380 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:31.227385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:31.227439 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:31.227515 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:31.228269 (+ 754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59590: BlockingRecv error: recv got EOF from 127.0.0.1:59590 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 02:44:31.229120 26151 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5cc)
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:31.230603 26151 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:31.231837 26151 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:44:31.244666 3039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:31.233132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59604 (local address 127.25.137.193:33185)
0808 02:44:31.233275 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:31.233280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:31.233492 (+ 212us) server_negotiation.cc:407] Connection header received
0808 02:44:31.233607 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:31.233610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:31.233667 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:31.233741 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:31.234651 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.235405 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.236558 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.236967 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.237103 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:31.240301 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:31.240306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:31.240309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:31.240352 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:31.242676 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.243218 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.243222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.243224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.243355 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.243642 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.243645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.243647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.243932 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:31.244120 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:31.244303 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 02:44:31.244451 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 02:44:31.249117 3039 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:31.247474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59620 (local address 127.25.137.193:33185)
0808 02:44:31.247659 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:31.247664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:31.247937 (+ 273us) server_negotiation.cc:407] Connection header received
0808 02:44:31.248081 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:31.248086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:31.248161 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:31.248239 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:31.248879 (+ 640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59620: BlockingRecv error: recv got EOF from 127.0.0.1:59620 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 02:44:31.249380 26151 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157019169307-26151-0/krb5kdc/krb5cc' of type FILE
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:31.250885 26151 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:31.252168 26151 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:44:31.265937 3039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:31.253442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59622 (local address 127.25.137.193:33185)
0808 02:44:31.253618 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:31.253623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:31.253917 (+ 294us) server_negotiation.cc:407] Connection header received
0808 02:44:31.254070 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:31.254075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:31.254142 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:31.254228 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:31.255101 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.255814 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.257077 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.257483 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.257619 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:31.260950 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:31.260955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:31.260958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:31.260990 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:31.263468 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.264066 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.264071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.264075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.264231 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.264634 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.264638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.264640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.264923 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:31.265239 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:31.265426 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 02:44:31.265629 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:44:31.267396 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 2405
I20260808 02:44:31.278986 3039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:31.268050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59634 (local address 127.25.137.193:33185)
0808 02:44:31.268188 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:31.268192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:31.268559 (+ 367us) server_negotiation.cc:407] Connection header received
0808 02:44:31.268713 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:31.268718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:31.268786 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:31.268880 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:31.269792 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.270563 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.271757 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.272197 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.272330 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:31.274290 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:31.274294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:31.274298 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:31.274327 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:31.276692 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.277346 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.277350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.277353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.277500 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.277837 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.277840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.277842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.278085 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:31.278286 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:31.278501 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 02:44:31.278666 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:44:31.279493 2591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:31.361997 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 2405
I20260808 02:44:31.381235 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 2608
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:44:31.396569 3057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:31.382124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49992 (local address 127.25.137.194:46173)
0808 02:44:31.382700 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:31.382708 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:31.382741 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:44:31.382810 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:31.382817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:31.383058 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:31.383239 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:31.384280 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.385551 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.386771 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.387331 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.387425 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:31.391272 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:31.391284 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:31.391307 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:31.391348 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:31.394102 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.394771 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.394777 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.394779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.394909 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.395190 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.395193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.395195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.395533 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:31.395760 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:31.396058 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 02:44:31.396264 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":273,"threads_started":1}
I20260808 02:44:31.397949 2801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:31.481441 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 2608
I20260808 02:44:31.505411 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 2817
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[2241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:44:31.520834 3059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:31.506335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53028 (local address 127.25.137.195:44547)
0808 02:44:31.507042 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:31.507048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:31.507079 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:44:31.507139 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:31.507146 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:31.507366 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:31.507510 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:31.508596 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.509691 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.510931 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:31.511501 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:31.511617 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:31.515483 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:31.515495 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:31.515510 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:31.515543 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:31.518217 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.518883 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.518890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.518893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.519014 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:31.519377 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:31.519381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:31.519384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:31.519714 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:31.520021 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:31.520295 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 02:44:31.520549 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":333,"threads_started":1}
I20260808 02:44:31.522116 3019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:31.605466 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 2817
I20260808 02:44:31.626895 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 2264
I20260808 02:44:31.628123 2359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:31.730917 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 2264
2026-08-08T02:44:31Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4083 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[3068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[3068](info): set up 2 sockets
Aug 08 02:44:31 dist-test-slave-k5rr krb5kdc[3068](info): commencing operation
krb5kdc: starting...
W20260808 02:44:32.948804 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.004s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:44:33 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.254.keytab.
Entry for principal oryx/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.254.keytab.
I20260808 02:44:33.191915 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:34061
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.254.keytab
--principal=oryx/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:34061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:33.522311 3085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:33.522599 3085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:33.522727 3085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:33.529860 3085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:33.529959 3085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:33.530030 3085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:33.530100 3085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:33.530155 3085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:33.539736 3085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:34061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.254.keytab
--principal=oryx/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:34061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:33.541059 3085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:33.542387 3085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:33.554008 3097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:33.554031 3095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:33.554208 3093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:33.554916 3085 server_base.cc:1034] running on GCE node
I20260808 02:44:33.555698 3085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:33.557024 3085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:33.558238 3085 hybrid_clock.cc:648] HybridClock initialized: now 1786157073558194 us; error 32 us; skew 500 ppm
I20260808 02:44:33.558585 3085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:33 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157073, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:33.561673 3085 init.cc:377] Logged in from keytab as oryx/127.25.137.254@KRBTEST.COM (short username oryx)
I20260808 02:44:33.563097 3085 webserver.cc:460] Webserver started at http://127.25.137.254:42689/ using document root <none> and password file <none>
I20260808 02:44:33.563751 3085 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:33.563865 3085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:33.564157 3085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:33.567085 3085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "62b3c63d1651463a974424f583236de8"
format_stamp: "Formatted at 2026-08-08 02:44:33 on dist-test-slave-k5rr"
server_key: "9676068a3db743738e93cd086e07d3e6"
server_key_iv: "0b482df69cd3d122b8587846c685a0d1"
server_key_version: "encryptionkey@0"
I20260808 02:44:33.567836 3085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "62b3c63d1651463a974424f583236de8"
format_stamp: "Formatted at 2026-08-08 02:44:33 on dist-test-slave-k5rr"
server_key: "9676068a3db743738e93cd086e07d3e6"
server_key_iv: "0b482df69cd3d122b8587846c685a0d1"
server_key_version: "encryptionkey@0"
I20260808 02:44:33.572806 3085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:44:33.576180 3107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:33.577493 3085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:44:33.577648 3085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "62b3c63d1651463a974424f583236de8"
format_stamp: "Formatted at 2026-08-08 02:44:33 on dist-test-slave-k5rr"
server_key: "9676068a3db743738e93cd086e07d3e6"
server_key_iv: "0b482df69cd3d122b8587846c685a0d1"
server_key_version: "encryptionkey@0"
I20260808 02:44:33.577818 3085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:33.599517 3085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:33.603443 3085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:33.603706 3085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:33.621327 3085 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:34061
I20260808 02:44:33.621357 3189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:34061 every 8 connection(s)
I20260808 02:44:33.622634 3085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:33.624972 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 3085
I20260808 02:44:33.625241 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:33.626101 26151 external_mini_cluster.cc:1428] Setting key bc5c2ca0179d6959a4b9e722442df9cc
I20260808 02:44:33.626945 3190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 02:44:33 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:33.641772 3190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8: Bootstrap starting.
I20260808 02:44:33.645108 3190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:33.646363 3190 log.cc:824] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:33.648809 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:33.628204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.25.137.254:34061)
0808 02:44:33.628953 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:33.628964 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:33.629034 (+ 70us) server_negotiation.cc:407] Connection header received
0808 02:44:33.630539 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:33.630560 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:33.630934 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:33.631254 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:33.633122 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:33.634465 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:33.636177 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:33.636833 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:33.636969 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:33.641688 (+ 4719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:33.641696 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:33.641711 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:33.641752 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:33.644633 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:33.645463 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:33.645470 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:33.645476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:33.645658 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:33.646049 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:33.646052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:33.646054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:33.646593 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:33.646851 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:33.647386 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 02:44:33.647840 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":333,"threads_started":1}
I20260808 02:44:33.649549 3190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8: No bootstrap required, opened a new log
I20260808 02:44:33.656890 3190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } }
I20260808 02:44:33.657318 3190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:33.657415 3190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62b3c63d1651463a974424f583236de8, State: Initialized, Role: FOLLOWER
I20260808 02:44:33.657950 3190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } }
I20260808 02:44:33.658178 3190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:33.658316 3190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:33.658475 3190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:33.660619 3190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } }
I20260808 02:44:33.661190 3190 leader_election.cc:302] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62b3c63d1651463a974424f583236de8; no voters:
I20260808 02:44:33.661805 3190 leader_election.cc:288] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:33.661926 3196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:33.662659 3196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 1 LEADER]: Becoming Leader. State: Replica: 62b3c63d1651463a974424f583236de8, State: Running, Role: LEADER
I20260808 02:44:33.663168 3196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } }
I20260808 02:44:33.663813 3190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:33.665979 3203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62b3c63d1651463a974424f583236de8. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } }
I20260808 02:44:33.666507 3203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:33.665908 3197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } }
I20260808 02:44:33.666699 3197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:33.668390 3215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:33.672986 3215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:33.681181 3215 catalog_manager.cc:1347] Generated new cluster ID: 8e115c5b06734bc486ed5cec4912f3fa
I20260808 02:44:33.681303 3215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:33.699674 3215 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:33.701097 3215 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:33.709498 3215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8: Generated new TSK 0
I20260808 02:44:33.710368 3215 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab.
Entry for principal oryx/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab.
I20260808 02:44:33.774700 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34061
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab
--principal=oryx/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:34.102761 3235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:34.103076 3235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:34.103216 3235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:34.110529 3235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:34.110662 3235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:34.110826 3235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:44:34.120553 3235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab
--principal=oryx/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:34.121908 3235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:34.123193 3235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:34.135582 3240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:34.135622 3248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:34.135663 3245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:34.136394 3235 server_base.cc:1034] running on GCE node
I20260808 02:44:34.136932 3235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:34.137838 3235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:34.139055 3235 hybrid_clock.cc:648] HybridClock initialized: now 1786157074139006 us; error 44 us; skew 500 ppm
I20260808 02:44:34.139355 3235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:34 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157074, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:34.142521 3235 init.cc:377] Logged in from keytab as oryx/127.25.137.193@KRBTEST.COM (short username oryx)
I20260808 02:44:34.144003 3235 webserver.cc:460] Webserver started at http://127.25.137.193:40781/ using document root <none> and password file <none>
I20260808 02:44:34.144654 3235 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:34.144773 3235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:34.145109 3235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:34.148129 3235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "645595ba49ff4a60ac33aca9c703bc0a"
format_stamp: "Formatted at 2026-08-08 02:44:34 on dist-test-slave-k5rr"
server_key: "c9e1d497197ff33c085186c53c242f57"
server_key_iv: "6f8f7e52fdbb279e40d8d9a98d045bb8"
server_key_version: "encryptionkey@0"
I20260808 02:44:34.148944 3235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "645595ba49ff4a60ac33aca9c703bc0a"
format_stamp: "Formatted at 2026-08-08 02:44:34 on dist-test-slave-k5rr"
server_key: "c9e1d497197ff33c085186c53c242f57"
server_key_iv: "6f8f7e52fdbb279e40d8d9a98d045bb8"
server_key_version: "encryptionkey@0"
I20260808 02:44:34.154098 3235 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:44:34.157399 3258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:34.158667 3235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:44:34.158843 3235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "645595ba49ff4a60ac33aca9c703bc0a"
format_stamp: "Formatted at 2026-08-08 02:44:34 on dist-test-slave-k5rr"
server_key: "c9e1d497197ff33c085186c53c242f57"
server_key_iv: "6f8f7e52fdbb279e40d8d9a98d045bb8"
server_key_version: "encryptionkey@0"
I20260808 02:44:34.159022 3235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:34.176994 3235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:34.181200 3235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:34.181463 3235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:34.182468 3235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:34.183809 3235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:34.183889 3235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:34.184012 3235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:34.184067 3235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:34.215569 3235 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:46793
I20260808 02:44:34.215607 3443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:46793 every 8 connection(s)
I20260808 02:44:34.216913 3235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:44:34.219923 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 3235
I20260808 02:44:34.220129 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:44:34.220944 26151 external_mini_cluster.cc:1428] Setting key e3cbfebd3355d916227bacef160e057d
Aug 08 02:44:34 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157074, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:34.238030 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:34.219336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:59827 (local address 127.25.137.254:34061)
0808 02:44:34.219811 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:34.219817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:34.220953 (+ 1136us) server_negotiation.cc:407] Connection header received
0808 02:44:34.223076 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:34.223080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:34.223143 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:34.223232 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:34.225226 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:34.225954 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:34.228004 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:34.228455 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:34.228556 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:34.232475 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:34.232480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:34.232483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:34.232517 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:34.235020 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:34.235744 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:34.235748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:34.235750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:34.235879 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:34.236292 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:34.236295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:34.236298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:34.236561 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:34.236696 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:34.237433 (+ 737us) server_negotiation.cc:299] Negotiation successful
0808 02:44:34.237659 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":227,"threads_started":1}
I20260808 02:44:34.239049 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:34.219893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.193:59827)
0808 02:44:34.220730 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:34.220812 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:34.222797 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:34.223506 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:34.223517 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:34.224226 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:34.225040 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:34.225068 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:34.226136 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:34.226141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:34.227828 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:34.227844 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:34.228682 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:34.228690 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:34.228957 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:34.229762 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:34.229801 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:34.232246 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:34.235176 (+ 2930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:34.235185 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:34.235201 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:34.235614 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:34.235983 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:34.235988 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:34.235992 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:34.236177 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:34.236687 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:34.236694 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:34.237052 (+ 358us) client_negotiation.cc:769] Sending connection context
0808 02:44:34.237313 (+ 261us) client_negotiation.cc:240] Negotiation successful
0808 02:44:34.237833 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":302,"threads_started":1}
I20260808 02:44:34.240720 3447 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34061
I20260808 02:44:34.241129 3447 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:34.241974 3447 heartbeater.cc:511] Master 127.25.137.254:34061 requested a full tablet report, sending...
I20260808 02:44:34.244437 3133 ts_manager.cc:194] Registered new tserver with Master: 645595ba49ff4a60ac33aca9c703bc0a (127.25.137.193:46793)
WARNING: no policy specified for oryx/127.25.137.194@KRBTEST.COM; defaulting to no policy
I20260808 02:44:34.246609 3133 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:59827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab.
Entry for principal oryx/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab.
I20260808 02:44:34.279779 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34061
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab
--principal=oryx/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:34.616552 3459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:34.616853 3459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:34.616968 3459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:34.626405 3459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:34.626546 3459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:34.626761 3459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:44:34.642376 3459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab
--principal=oryx/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:34.643914 3459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:34.645299 3459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:34.658459 3468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:34.658417 3469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:34.658485 3476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:34.659121 3459 server_base.cc:1034] running on GCE node
I20260808 02:44:34.659714 3459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:34.660669 3459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:34.661903 3459 hybrid_clock.cc:648] HybridClock initialized: now 1786157074661852 us; error 47 us; skew 500 ppm
I20260808 02:44:34.662245 3459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:34 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157074, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:34.665481 3459 init.cc:377] Logged in from keytab as oryx/127.25.137.194@KRBTEST.COM (short username oryx)
I20260808 02:44:34.666989 3459 webserver.cc:460] Webserver started at http://127.25.137.194:41713/ using document root <none> and password file <none>
I20260808 02:44:34.667797 3459 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:34.667936 3459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:34.668277 3459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:34.671454 3459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "b2794284317c4a899881ef7bc6ce223f"
format_stamp: "Formatted at 2026-08-08 02:44:34 on dist-test-slave-k5rr"
server_key: "368cc69bee90d17faed63a2e0b166b67"
server_key_iv: "3d1ea8b50e02d8887057333b67f494d3"
server_key_version: "encryptionkey@0"
I20260808 02:44:34.672243 3459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "b2794284317c4a899881ef7bc6ce223f"
format_stamp: "Formatted at 2026-08-08 02:44:34 on dist-test-slave-k5rr"
server_key: "368cc69bee90d17faed63a2e0b166b67"
server_key_iv: "3d1ea8b50e02d8887057333b67f494d3"
server_key_version: "encryptionkey@0"
I20260808 02:44:34.677603 3459 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:44:34.681120 3486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:34.682564 3459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:34.682713 3459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "b2794284317c4a899881ef7bc6ce223f"
format_stamp: "Formatted at 2026-08-08 02:44:34 on dist-test-slave-k5rr"
server_key: "368cc69bee90d17faed63a2e0b166b67"
server_key_iv: "3d1ea8b50e02d8887057333b67f494d3"
server_key_version: "encryptionkey@0"
I20260808 02:44:34.682902 3459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:34.708968 3459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:34.713353 3459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:34.713626 3459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:34.714629 3459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:34.715992 3459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:34.716068 3459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:34.716177 3459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:34.716228 3459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:34.746815 3459 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:44887
I20260808 02:44:34.746855 3658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:44887 every 8 connection(s)
I20260808 02:44:34.748045 3459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:44:34.755051 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 3459
I20260808 02:44:34.755244 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:44:34.755664 26151 external_mini_cluster.cc:1428] Setting key 1ca6ecb1c4bafb5584fc1004213c414d
Aug 08 02:44:34 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157074, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:34.768362 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:34.750582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:51965 (local address 127.25.137.254:34061)
0808 02:44:34.751039 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:34.751044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:34.752033 (+ 989us) server_negotiation.cc:407] Connection header received
0808 02:44:34.753889 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:34.753894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:34.753972 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:34.754079 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:34.755630 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:34.756385 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:34.758204 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:34.758628 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:34.758738 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:34.762328 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:34.762333 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:34.762337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:34.762370 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:34.765052 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:34.765940 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:34.765945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:34.765947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:34.766082 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:34.766533 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:34.766536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:34.766538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:34.766794 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:34.766912 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:34.767739 (+ 827us) server_negotiation.cc:299] Negotiation successful
0808 02:44:34.767965 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":237,"threads_started":1}
I20260808 02:44:34.769189 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:34.751128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.194:51965)
0808 02:44:34.751868 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:34.751914 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:34.753611 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:34.754317 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:34.754325 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:34.754869 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:34.755466 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:34.755485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:34.756530 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:34.756535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:34.758059 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:34.758077 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:34.758852 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:34.758860 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:34.759079 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:34.759686 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:34.759709 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:34.762106 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:34.765202 (+ 3096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:34.765210 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:34.765225 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:34.765790 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:34.766184 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:34.766188 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:34.766191 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:34.766435 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:34.766915 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:34.766922 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:34.767325 (+ 403us) client_negotiation.cc:769] Sending connection context
0808 02:44:34.767568 (+ 243us) client_negotiation.cc:240] Negotiation successful
0808 02:44:34.767996 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":288,"threads_started":1}
I20260808 02:44:34.770646 3659 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34061
I20260808 02:44:34.770973 3659 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:34.771852 3659 heartbeater.cc:511] Master 127.25.137.254:34061 requested a full tablet report, sending...
I20260808 02:44:34.773469 3133 ts_manager.cc:194] Registered new tserver with Master: b2794284317c4a899881ef7bc6ce223f (127.25.137.194:44887)
I20260808 02:44:34.774308 3133 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:51965
WARNING: no policy specified for oryx/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab.
Entry for principal oryx/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab.
I20260808 02:44:34.811542 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34061
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab
--principal=oryx/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:35.134799 3668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:35.135066 3668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:35.135205 3668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:35.142241 3668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:35.142344 3668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:35.142488 3668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:44:35.151917 3668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab
--principal=oryx/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:35.153211 3668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:35.154520 3668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:35.167459 3676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:35.167541 3677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:35.167582 3680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:35.168233 3668 server_base.cc:1034] running on GCE node
I20260808 02:44:35.168892 3668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:35.169885 3668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:35.171088 3668 hybrid_clock.cc:648] HybridClock initialized: now 1786157075171036 us; error 37 us; skew 500 ppm
I20260808 02:44:35.171402 3668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:35 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157075, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:35.176966 3668 init.cc:377] Logged in from keytab as oryx/127.25.137.195@KRBTEST.COM (short username oryx)
I20260808 02:44:35.178601 3668 webserver.cc:460] Webserver started at http://127.25.137.195:38683/ using document root <none> and password file <none>
I20260808 02:44:35.179280 3668 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:35.179392 3668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:35.179679 3668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:35.182732 3668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "cee9bde7a81745e09b5449e36a323e39"
format_stamp: "Formatted at 2026-08-08 02:44:35 on dist-test-slave-k5rr"
server_key: "782f94352a9cdf1cb34ab441d05cae2b"
server_key_iv: "a804700ac3890e0d56ec126450f8bfc4"
server_key_version: "encryptionkey@0"
I20260808 02:44:35.183518 3668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "cee9bde7a81745e09b5449e36a323e39"
format_stamp: "Formatted at 2026-08-08 02:44:35 on dist-test-slave-k5rr"
server_key: "782f94352a9cdf1cb34ab441d05cae2b"
server_key_iv: "a804700ac3890e0d56ec126450f8bfc4"
server_key_version: "encryptionkey@0"
I20260808 02:44:35.189116 3668 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:44:35.192560 3693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:35.193887 3668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:44:35.194046 3668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "cee9bde7a81745e09b5449e36a323e39"
format_stamp: "Formatted at 2026-08-08 02:44:35 on dist-test-slave-k5rr"
server_key: "782f94352a9cdf1cb34ab441d05cae2b"
server_key_iv: "a804700ac3890e0d56ec126450f8bfc4"
server_key_version: "encryptionkey@0"
I20260808 02:44:35.194245 3668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:35.212879 3668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:35.217074 3668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:35.217347 3668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:35.218268 3668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:35.219695 3668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:35.219774 3668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:35.219892 3668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:35.219942 3668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:35.249845 3447 heartbeater.cc:503] Master 127.25.137.254:34061 was elected leader, sending a full tablet report...
I20260808 02:44:35.251030 3668 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:33541
I20260808 02:44:35.251072 3878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:33541 every 8 connection(s)
I20260808 02:44:35.252368 3668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:44:35.254588 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 3668
I20260808 02:44:35.254794 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:44:35.255353 26151 external_mini_cluster.cc:1428] Setting key 5205be1f00b6f53699609e6bfa768401
Aug 08 02:44:35 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157075, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:35.272493 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.254870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:56565 (local address 127.25.137.254:34061)
0808 02:44:35.255095 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:35.255101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:35.256262 (+ 1161us) server_negotiation.cc:407] Connection header received
0808 02:44:35.258415 (+ 2153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:35.258419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:35.258494 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:35.258574 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:35.259957 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.260690 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.262210 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.262702 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.262835 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.266331 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:35.266337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:35.266341 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:35.266374 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:35.269145 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.270167 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.270173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.270177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.270316 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.270782 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.270788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.270792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.271086 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:35.271228 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:35.271943 (+ 715us) server_negotiation.cc:299] Negotiation successful
0808 02:44:35.272169 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:44:35.273459 3886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.255313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.195:56565)
0808 02:44:35.256085 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:35.256134 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:35.258149 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:35.258770 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:35.258778 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:35.259274 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:35.259811 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.259828 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.260800 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.260804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.262055 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.262068 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.262912 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.262917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.263124 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.263729 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:35.263755 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:35.266103 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:35.269285 (+ 3182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.269294 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.269311 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.269870 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.270422 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.270426 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.270429 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.270662 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.271210 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:35.271218 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:35.271572 (+ 354us) client_negotiation.cc:769] Sending connection context
0808 02:44:35.271809 (+ 237us) client_negotiation.cc:240] Negotiation successful
0808 02:44:35.272312 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":311,"threads_started":1}
I20260808 02:44:35.274700 3879 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34061
I20260808 02:44:35.275035 3879 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:35.275774 3879 heartbeater.cc:511] Master 127.25.137.254:34061 requested a full tablet report, sending...
I20260808 02:44:35.277181 3133 ts_manager.cc:194] Registered new tserver with Master: cee9bde7a81745e09b5449e36a323e39 (127.25.137.195:33541)
I20260808 02:44:35.278029 3133 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:56565
I20260808 02:44:35.282903 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.137.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.253.keytab.
Entry for principal oryx/127.25.137.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.253.keytab.
I20260808 02:44:35.336059 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.253:45953
--webserver_interface=127.25.137.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.253.keytab
--principal=oryx/127.25.137.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:34061,127.25.137.253:45953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:35.669194 3890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:35.669523 3890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:35.669660 3890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:35.676906 3890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:35.677085 3890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:35.677173 3890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:35.677261 3890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:35.677325 3890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:35.687579 3890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:34061,127.25.137.253:45953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.253.keytab
--principal=oryx/127.25.137.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.253:45953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.137.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
W20260808 02:44:35.687964 3890 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.25.137.254:34061,127.25.137.253:45953'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 02:44:35.689039 3890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:35.690399 3890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:35.701998 3897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:35.702152 3900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:35.702140 3896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:35.703024 3890 server_base.cc:1034] running on GCE node
I20260808 02:44:35.703807 3890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:35.705308 3890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:35.706549 3890 hybrid_clock.cc:648] HybridClock initialized: now 1786157075706470 us; error 51 us; skew 500 ppm
I20260808 02:44:35.706924 3890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:35 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157075, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:35.710145 3890 init.cc:377] Logged in from keytab as oryx/127.25.137.253@KRBTEST.COM (short username oryx)
I20260808 02:44:35.711653 3890 webserver.cc:460] Webserver started at http://127.25.137.253:34367/ using document root <none> and password file <none>
I20260808 02:44:35.712335 3890 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:35.712424 3890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:35.712685 3890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:35.715672 3890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data/instance:
uuid: "90edc8737f924dfe82035aebcbc31b12"
format_stamp: "Formatted at 2026-08-08 02:44:35 on dist-test-slave-k5rr"
server_key: "dd85ba0cadf34a064098650888e8ab2d"
server_key_iv: "98f89ffb6f6eba7c7e160b6a7261ea66"
server_key_version: "encryptionkey@0"
I20260808 02:44:35.716411 3890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/wal/instance:
uuid: "90edc8737f924dfe82035aebcbc31b12"
format_stamp: "Formatted at 2026-08-08 02:44:35 on dist-test-slave-k5rr"
server_key: "dd85ba0cadf34a064098650888e8ab2d"
server_key_iv: "98f89ffb6f6eba7c7e160b6a7261ea66"
server_key_version: "encryptionkey@0"
I20260808 02:44:35.721634 3890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260808 02:44:35.724997 3914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:35.726318 3890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:35.726464 3890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/wal
uuid: "90edc8737f924dfe82035aebcbc31b12"
format_stamp: "Formatted at 2026-08-08 02:44:35 on dist-test-slave-k5rr"
server_key: "dd85ba0cadf34a064098650888e8ab2d"
server_key_iv: "98f89ffb6f6eba7c7e160b6a7261ea66"
server_key_version: "encryptionkey@0"
I20260808 02:44:35.726653 3890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:35.745451 3890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:35.749565 3890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:35.749814 3890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:35.769676 3890 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.253:45953
I20260808 02:44:35.769711 4002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.253:45953 every 8 connection(s)
I20260808 02:44:35.771054 3890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
I20260808 02:44:35.774801 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:35.777222 3659 heartbeater.cc:503] Master 127.25.137.254:34061 was elected leader, sending a full tablet report...
I20260808 02:44:35.779551 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 3890
I20260808 02:44:35.782513 4003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:44:35 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157075, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.253@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
Aug 08 02:44:35 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.137.253@KRBTEST.COM
I20260808 02:44:35.801563 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.786347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60812 (local address 127.25.137.254:34061)
0808 02:44:35.786861 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:35.786866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:35.787340 (+ 474us) server_negotiation.cc:407] Connection header received
0808 02:44:35.787495 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:35.787501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:35.787589 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:35.787707 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:35.788832 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.789593 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.790964 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.791364 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.791473 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.794877 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:35.794883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:35.794887 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:35.794923 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:35.798319 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.799211 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.799217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.799221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.799344 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.799735 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.799740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.799744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.800110 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:35.800244 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:35.800936 (+ 692us) server_negotiation.cc:299] Negotiation successful
0808 02:44:35.801193 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":220,"threads_started":1}
I20260808 02:44:35.802627 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.786324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:60812)
0808 02:44:35.787148 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:35.787192 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:35.787367 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:35.787766 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:35.787772 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:35.788392 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:35.788677 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.788687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.789726 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.789730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.790828 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.790840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.791585 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.791593 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.791735 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.792313 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:35.792335 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:35.794678 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:35.798495 (+ 3817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.798505 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.798513 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.799104 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.799448 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.799453 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.799455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.799623 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.800245 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:35.800252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:35.800580 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 02:44:35.800821 (+ 241us) client_negotiation.cc:240] Negotiation successful
0808 02:44:35.801353 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":3,"thread_start_us":446,"threads_started":1}
I20260808 02:44:35.802677 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.781932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57922 (local address 127.25.137.253:45953)
0808 02:44:35.782715 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:35.782731 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:35.782788 (+ 57us) server_negotiation.cc:407] Connection header received
0808 02:44:35.784535 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:35.784557 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:35.784954 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:35.785331 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:35.786959 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.788643 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.789879 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.790504 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.790684 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.795400 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:35.795411 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:35.795427 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:35.795475 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:35.798999 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.799711 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.799718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.799720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.799866 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.800289 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.800293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.800295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.800693 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:35.801036 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:35.801392 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 02:44:35.801739 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":332,"threads_started":1}
I20260808 02:44:35.804679 4003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:44:35 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157075, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.253@KRBTEST.COM for oryx/127.25.137.253@KRBTEST.COM
I20260808 02:44:35.816174 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.805377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:45953 (local address 127.0.0.1:57928)
0808 02:44:35.805610 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:35.805633 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:35.805728 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:35.805988 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:35.805992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:35.806331 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:35.806453 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.806461 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.807373 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.807378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.808221 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.808231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.808863 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.808868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.809073 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.809622 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:35.809637 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:35.811538 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:35.814044 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.814049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.814053 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.814390 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.814703 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.814708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.814712 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.814860 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.815332 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:35.815336 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:35.815454 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 02:44:35.815563 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 02:44:35.815822 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260808 02:44:35.816239 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.805634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57928 (local address 127.25.137.253:45953)
0808 02:44:35.805776 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:35.805781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:35.805796 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:44:35.805836 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:35.805840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:35.805886 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:35.805957 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:35.806576 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.807239 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.808353 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.808733 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.808840 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.811705 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:35.811709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:35.811712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:35.811739 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:35.813929 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.814491 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.814496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.814498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.814597 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.814959 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.814964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.814967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.815224 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:35.815353 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:35.815650 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 02:44:35.815952 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":49}
I20260808 02:44:35.820844 4003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: Bootstrap starting.
I20260808 02:44:35.823992 4003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:35.825073 4003 log.cc:824] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:35.827982 4003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: No bootstrap required, opened a new log
I20260808 02:44:35.835590 4003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } }
I20260808 02:44:35.835919 4003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:35.836026 4003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90edc8737f924dfe82035aebcbc31b12, State: Initialized, Role: FOLLOWER
I20260808 02:44:35.836612 4003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } }
I20260808 02:44:35.837466 4011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } } }
I20260808 02:44:35.837746 4011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:44:35.838495 4003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:44:35.847355 4025 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:44:35.847483 4025 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:44:35.850983 3890 master_runner.cc:428] Detected that this master 90edc8737f924dfe82035aebcbc31b12 is joining an existing cluster
I20260808 02:44:35.851070 3890 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.137.253:45953
I20260808 02:44:35.872597 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.855246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60824 (local address 127.25.137.254:34061)
0808 02:44:35.855432 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:35.855438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:35.855780 (+ 342us) server_negotiation.cc:407] Connection header received
0808 02:44:35.856070 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:35.856075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:35.856177 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:35.856261 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:35.858216 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.859409 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.861645 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.862384 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.862505 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.865460 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:35.865467 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:35.865472 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:35.865514 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:35.869185 (+ 3671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.870093 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.870099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.870102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.870257 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.870749 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.870754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.870757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.871136 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:35.871272 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:35.871997 (+ 725us) server_negotiation.cc:299] Negotiation successful
0808 02:44:35.872250 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:44:35.873754 4032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.854927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:60824)
0808 02:44:35.855678 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:35.855702 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:35.855915 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:35.856520 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:35.856525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:35.857395 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:35.858029 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.858045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.859614 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.859619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.861456 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.861473 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.862883 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.862890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.863085 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.863837 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:35.863867 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:35.865218 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:35.869356 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.869361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.869365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.869936 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.870414 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.870419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.870423 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.870611 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.871361 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:35.871366 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:35.871653 (+ 287us) client_negotiation.cc:769] Sending connection context
0808 02:44:35.872877 (+ 1224us) client_negotiation.cc:240] Negotiation successful
0808 02:44:35.873248 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":437,"threads_started":1}
I20260808 02:44:35.892653 4034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.856074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:45953 (local address 127.0.0.1:57932)
0808 02:44:35.868195 (+ 12121us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:35.868223 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:35.868365 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:35.868990 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:35.868996 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:35.869690 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:35.870054 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.870066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.871589 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.871597 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.873054 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.873069 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.874373 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.874380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.874533 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.885282 (+ 10749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:35.885320 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:35.886499 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:35.889820 (+ 3321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.889825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.889828 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.890310 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.890689 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.890697 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.890701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.890860 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.891579 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:35.891584 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:35.891724 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 02:44:35.891889 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 02:44:35.892177 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11992,"thread_start_us":11827,"threads_started":1}
I20260808 02:44:35.893723 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.855925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57932 (local address 127.25.137.253:45953)
0808 02:44:35.856114 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:35.856120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:35.868322 (+ 12202us) server_negotiation.cc:407] Connection header received
0808 02:44:35.868546 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:35.868552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:35.868620 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:35.868696 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:35.870201 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.871281 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.873202 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.873947 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.874038 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.886706 (+ 12668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:35.886712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:35.886721 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:35.886757 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:35.889686 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.890430 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.890435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.890438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.890582 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.890968 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.890972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.890974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.891341 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:35.891527 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:35.893161 (+ 1634us) server_negotiation.cc:299] Negotiation successful
0808 02:44:35.893386 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
WARNING: no policy specified for oryx/127.25.137.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.252@KRBTEST.COM" created.
Aug 08 02:44:35 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157073, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.254@KRBTEST.COM for oryx/127.25.137.253@KRBTEST.COM
I20260808 02:44:35.910699 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.880864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:45953 (local address 127.0.0.1:57936)
0808 02:44:35.892951 (+ 12087us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:35.893061 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:35.893264 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:35.894182 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:35.894191 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:35.895077 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:35.895457 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.895472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.897297 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.897303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.898890 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.898906 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.899751 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.899758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:35.900010 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.900683 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:35.900708 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:35.903486 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:35.907686 (+ 4200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.907702 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.907711 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.908288 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.908715 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:35.908720 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:35.908723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:35.908919 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:35.909563 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:35.909572 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:35.909779 (+ 207us) client_negotiation.cc:769] Sending connection context
0808 02:44:35.909996 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 02:44:35.910305 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11780,"thread_start_us":12566,"threads_started":1}
I20260808 02:44:35.911033 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.880939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57936 (local address 127.25.137.253:45953)
0808 02:44:35.889961 (+ 9022us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:35.889966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:35.893139 (+ 3173us) server_negotiation.cc:407] Connection header received
0808 02:44:35.893500 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:35.893505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:35.893592 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:35.893717 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:35.895682 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.897078 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.899050 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:35.899572 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:35.899669 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:35.903723 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:35.903730 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:35.903735 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:35.903775 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:35.907532 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.908420 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.908426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.908430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.908596 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:35.909053 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:35.909059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:35.909062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:35.909427 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:35.909584 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:35.910114 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 02:44:35.910335 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7603,"thread_start_us":6732,"threads_started":1}
I20260808 02:44:35.913470 3133 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.137.253:45953: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "90edc8737f924dfe82035aebcbc31b12"
member_type: NON_VOTER
last_known_addr {
host: "127.25.137.253"
port: 45953
}
attrs {
promote: true
}
}
dest_uuid: "62b3c63d1651463a974424f583236de8"
cas_config_opid_index: -1
I20260808 02:44:35.915107 3133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } }
I20260808 02:44:35.930323 3958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:35.929214 4043 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 90edc8737f924dfe82035aebcbc31b12 (127.25.137.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } }
I20260808 02:44:35.934527 3958 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 FOLLOWER]: Refusing update from remote peer 62b3c63d1651463a974424f583236de8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 02:44:35.935086 4043 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.137.253:45953
I20260808 02:44:35.935678 4044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } }
I20260808 02:44:35.936116 4044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:35.936157 3890 master.cc:537] Master@127.25.137.253:45953 shutting down...
I20260808 02:44:35.937810 4044 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.002s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.252.keytab.
Entry for principal oryx/127.25.137.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.252.keytab.
I20260808 02:44:35.941663 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.252:42319
--webserver_interface=127.25.137.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.252.keytab
--principal=oryx/127.25.137.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:34061,127.25.137.253:45953,127.25.137.252:42319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:35.944434 3111 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 02:44:35.946156 3111 consensus_peers.cc:487] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 -> Peer 90edc8737f924dfe82035aebcbc31b12 (127.25.137.253:45953): Couldn't send request to peer 90edc8737f924dfe82035aebcbc31b12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 02:44:35.951719 3890 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:35.952230 3890 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:35.952493 3890 tablet_replica.cc:330] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: stopping tablet replica
I20260808 02:44:35.970498 3890 master.cc:552] Master@127.25.137.253:45953 shutdown complete.
I20260808 02:44:35.970662 3890 master_runner.cc:305] Clearing existing system tablet
I20260808 02:44:35.971187 3890 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:35.971306 3890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:35.974661 3890 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.005s sys 0.000s
I20260808 02:44:35.976631 4052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:35.977043 3890 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:35.977535 3890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 02:44:35.977690 3890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/wal
uuid: "90edc8737f924dfe82035aebcbc31b12"
format_stamp: "Formatted at 2026-08-08 02:44:35 on dist-test-slave-k5rr"
server_key: "dd85ba0cadf34a064098650888e8ab2d"
server_key_iv: "98f89ffb6f6eba7c7e160b6a7261ea66"
server_key_version: "encryptionkey@0"
I20260808 02:44:35.984012 3890 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:35.991374 3890 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 02:44:35.991714 3890 log.cc:1196] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 02:44:35.992635 3890 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: Deleting consensus metadata
I20260808 02:44:35.992995 3890 master_runner.cc:315] Copying system tablet from 127.25.137.254:34061
I20260808 02:44:35.996105 3890 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: tablet copy: Beginning tablet copy session from remote peer at address 127.25.137.254:34061
I20260808 02:44:36.013768 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.997329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60840 (local address 127.25.137.254:34061)
0808 02:44:35.997563 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:35.997570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:35.998270 (+ 700us) server_negotiation.cc:407] Connection header received
0808 02:44:35.998454 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:35.998461 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:35.998573 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:35.998699 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:35.999800 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.000956 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.002921 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.003666 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.003780 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.006534 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:36.006542 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:36.006548 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:36.006593 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:36.010275 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.011152 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.011158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.011162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.011345 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.011870 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.011876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.011880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.012274 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:36.012418 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.013073 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.013326 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 02:44:36.015149 4064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:35.997436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:60840)
0808 02:44:35.998113 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:35.998140 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:35.998309 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:35.998761 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:35.998766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:35.999327 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:35.999628 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:35.999644 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.001192 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.001199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.002716 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.002734 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.004108 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.004115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.004341 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.005123 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:36.005151 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:36.006272 (+ 1121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:36.010449 (+ 4177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.010455 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.010460 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.010991 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.011531 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.011536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.011540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.011724 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.012532 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:36.012537 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:36.012732 (+ 195us) client_negotiation.cc:769] Sending connection context
0808 02:44:36.014258 (+ 1526us) client_negotiation.cc:240] Negotiation successful
0808 02:44:36.014643 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":294,"threads_started":1}
I20260808 02:44:36.016454 3162 tablet_copy_service.cc:140] P 62b3c63d1651463a974424f583236de8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 90edc8737f924dfe82035aebcbc31b12 ({username='oryx', principal='oryx/127.25.137.253@KRBTEST.COM'} at 127.0.0.1:60840)
I20260808 02:44:36.016925 3162 tablet_copy_service.cc:161] P 62b3c63d1651463a974424f583236de8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 90edc8737f924dfe82035aebcbc31b12 at {username='oryx', principal='oryx/127.25.137.253@KRBTEST.COM'} at 127.0.0.1:60840: session id = 90edc8737f924dfe82035aebcbc31b12-00000000000000000000000000000000
I20260808 02:44:36.022133 3162 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 02:44:36.024904 3890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 02:44:36.032121 3890 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: tablet copy: Starting download of 0 data blocks...
I20260808 02:44:36.032584 3890 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: tablet copy: Starting download of 1 WAL segments...
I20260808 02:44:36.036639 3890 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 02:44:36.041380 3162 tablet_copy_service.cc:342] P 62b3c63d1651463a974424f583236de8: Request end of tablet copy session 90edc8737f924dfe82035aebcbc31b12-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.137.253@KRBTEST.COM'} at 127.0.0.1:60840
I20260808 02:44:36.041814 3162 tablet_copy_service.cc:433] P 62b3c63d1651463a974424f583236de8: ending tablet copy session 90edc8737f924dfe82035aebcbc31b12-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 90edc8737f924dfe82035aebcbc31b12
I20260808 02:44:36.054322 3890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:36.057022 4066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:36.057541 4067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:36.067759 4071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:36.068675 3890 server_base.cc:1034] running on GCE node
I20260808 02:44:36.069186 3890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:36.069991 3890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:36.071230 3890 hybrid_clock.cc:648] HybridClock initialized: now 1786157076071128 us; error 58 us; skew 500 ppm
I20260808 02:44:36.071508 3890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:36 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157076, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:36.073948 3890 init.cc:377] Logged in from keytab as oryx/127.25.137.253@KRBTEST.COM (short username oryx)
I20260808 02:44:36.079059 3890 webserver.cc:460] Webserver started at http://127.25.137.253:40095/ using document root <none> and password file <none>
I20260808 02:44:36.079458 3890 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:36.079552 3890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:36.083096 3890 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 02:44:36.085188 4083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:36.085652 3890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 02:44:36.085848 3890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/wal
uuid: "90edc8737f924dfe82035aebcbc31b12"
format_stamp: "Formatted at 2026-08-08 02:44:35 on dist-test-slave-k5rr"
server_key: "dd85ba0cadf34a064098650888e8ab2d"
server_key_iv: "98f89ffb6f6eba7c7e160b6a7261ea66"
server_key_version: "encryptionkey@0"
I20260808 02:44:36.086118 3890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:36.099291 3890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:36.103595 3890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:36.124855 3890 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.253:45953
I20260808 02:44:36.124888 4174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.253:45953 every 8 connection(s)
I20260808 02:44:36.126543 3890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
I20260808 02:44:36.132306 4176 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:44:36.136786 4176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: Bootstrap starting.
I20260808 02:44:36.157507 4176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:44:36.158193 4176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: Bootstrap complete.
I20260808 02:44:36.159482 4176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } }
I20260808 02:44:36.159893 4176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 90edc8737f924dfe82035aebcbc31b12, State: Initialized, Role: LEARNER
I20260808 02:44:36.160188 4176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } }
I20260808 02:44:36.161527 4176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:36.161260 4181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } }
I20260808 02:44:36.161923 4181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:44:36.173545 4199 catalog_manager.cc:1259] Loaded cluster ID: 8e115c5b06734bc486ed5cec4912f3fa
I20260808 02:44:36.173666 4199 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: loading cluster ID for follower catalog manager: success
I20260808 02:44:36.177639 4199 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: acquiring CA information for follower catalog manager: success
I20260808 02:44:36.180418 4199 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:44:36.280468 3879 heartbeater.cc:503] Master 127.25.137.254:34061 was elected leader, sending a full tablet report...
W20260808 02:44:36.329136 4050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:36.329424 4050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:36.329522 4050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:36.336674 4050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:36.336776 4050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:36.336831 4050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:36.336896 4050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:36.336942 4050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:36.346469 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.340502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:45953 (local address 127.0.0.1:57946)
0808 02:44:36.340683 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:36.340700 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:36.340823 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:36.341674 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:36.341678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:36.341709 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:36.341934 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.341944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.343269 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.343274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.344522 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.344535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.345716 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.345722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.345925 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.345983 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 02:44:36.346071 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 02:44:36.346200 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 02:44:36.346554 4203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.340719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57946 (local address 127.25.137.253:45953)
0808 02:44:36.341347 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:36.341352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:36.341372 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:44:36.341434 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:36.341438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:36.341504 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:36.341613 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:36.342083 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.343125 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.344667 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.345556 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.345730 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.345930 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.346111 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.346239 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":352,"threads_started":1}
I20260808 02:44:36.347496 4128 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260808 02:44:36.347604 4050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:34061,127.25.137.253:45953,127.25.137.252:42319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.252.keytab
--principal=oryx/127.25.137.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.252:42319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.137.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:36.348042 4181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62b3c63d1651463a974424f583236de8. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } }
I20260808 02:44:36.348312 4181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:44:36.349146 4050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:36.350430 4050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:36.361285 4206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:36.362236 4207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:36.363168 4209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:36.363541 4050 server_base.cc:1034] running on GCE node
I20260808 02:44:36.364563 4050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:36.366001 4050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:36.367209 4050 hybrid_clock.cc:648] HybridClock initialized: now 1786157076367163 us; error 41 us; skew 500 ppm
I20260808 02:44:36.367553 4050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:36 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157076, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:36.370676 4050 init.cc:377] Logged in from keytab as oryx/127.25.137.252@KRBTEST.COM (short username oryx)
I20260808 02:44:36.372321 4050 webserver.cc:460] Webserver started at http://127.25.137.252:39327/ using document root <none> and password file <none>
I20260808 02:44:36.373062 4050 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:36.373193 4050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:36.373513 4050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:36.376760 4050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data/instance:
uuid: "56fdbdbfc708421c8fed116fef9abb70"
format_stamp: "Formatted at 2026-08-08 02:44:36 on dist-test-slave-k5rr"
server_key: "c02b9c0464c569ba8be0098e1878a63d"
server_key_iv: "c600b4f89ca3a2e62c636d76b4cd9fde"
server_key_version: "encryptionkey@0"
I20260808 02:44:36.377569 4050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/wal/instance:
uuid: "56fdbdbfc708421c8fed116fef9abb70"
format_stamp: "Formatted at 2026-08-08 02:44:36 on dist-test-slave-k5rr"
server_key: "c02b9c0464c569ba8be0098e1878a63d"
server_key_iv: "c600b4f89ca3a2e62c636d76b4cd9fde"
server_key_version: "encryptionkey@0"
I20260808 02:44:36.383150 4050 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:44:36.387019 4223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:36.388568 4050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:44:36.388717 4050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/wal
uuid: "56fdbdbfc708421c8fed116fef9abb70"
format_stamp: "Formatted at 2026-08-08 02:44:36 on dist-test-slave-k5rr"
server_key: "c02b9c0464c569ba8be0098e1878a63d"
server_key_iv: "c600b4f89ca3a2e62c636d76b4cd9fde"
server_key_version: "encryptionkey@0"
I20260808 02:44:36.388901 4050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:36.411969 4050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:36.415966 4050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:36.416211 4050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:36.435729 4050 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.252:42319
I20260808 02:44:36.435751 4296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.252:42319 every 8 connection(s)
I20260808 02:44:36.437160 4050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
I20260808 02:44:36.441020 4298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:36.444764 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 4050
I20260808 02:44:36.448179 4298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:44:36 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.137.252@KRBTEST.COM
Aug 08 02:44:36 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157076, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.252@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:36.468174 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.452150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60854 (local address 127.25.137.254:34061)
0808 02:44:36.452367 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:36.452373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:36.453205 (+ 832us) server_negotiation.cc:407] Connection header received
0808 02:44:36.453392 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:36.453396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:36.453459 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:36.453537 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:36.454920 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.455940 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.457429 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.458001 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.458134 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.461587 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:36.461594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:36.461599 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:36.461639 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:36.464802 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.465673 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.465678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.465681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.465874 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.466368 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.466372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.466375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.466726 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:36.466873 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.467594 (+ 721us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.467822 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:44:36.469277 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.446966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40696 (local address 127.25.137.252:42319)
0808 02:44:36.447783 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:36.447797 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:36.447842 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:44:36.449649 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:36.449671 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:36.450102 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:36.450471 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:36.452177 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.454220 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.455883 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.456709 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.456881 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.461265 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:36.461274 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:36.461285 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:36.461367 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:36.464783 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.465624 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.465632 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.465635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.465841 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.466374 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.466378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.466381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.467043 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:36.467307 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.467715 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.468002 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"spinlock_wait_cycles":14720,"thread_start_us":397,"threads_started":1}
I20260808 02:44:36.469297 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.452117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:60854)
0808 02:44:36.453054 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:36.453105 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:36.453277 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:36.453623 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:36.453630 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:36.454507 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:36.454760 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.454769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.456075 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.456078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.457277 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.457292 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.458206 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.458211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.458345 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.458877 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:36.458904 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:36.461368 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:36.464940 (+ 3572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.464951 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.464963 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.465558 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.466000 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.466006 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.466010 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.466243 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.466880 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:36.466886 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:36.467152 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 02:44:36.467488 (+ 336us) client_negotiation.cc:240] Negotiation successful
0808 02:44:36.468150 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":141,"thread_start_us":502,"threads_started":1}
I20260808 02:44:36.470798 4298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:44:36 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157076, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.252@KRBTEST.COM for oryx/127.25.137.253@KRBTEST.COM
I20260808 02:44:36.482734 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.471388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:45953 (local address 127.0.0.1:57956)
0808 02:44:36.471550 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:36.471567 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:36.471662 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:36.472014 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:36.472017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:36.472312 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:36.472446 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.472454 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.473527 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.473530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.474494 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.474506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.475154 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.475158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.475309 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.475785 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:36.475801 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:36.477738 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:36.480628 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.480632 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.480635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.480993 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.481346 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.481349 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.481351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.481495 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.482033 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:36.482037 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:36.482156 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 02:44:36.482259 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 02:44:36.482461 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 02:44:36.482827 4203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.471551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.25.137.253:45953)
0808 02:44:36.471747 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:36.471754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:36.471770 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:44:36.471827 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:36.471831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:36.471889 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:36.471986 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:36.472614 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.473415 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.474630 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.475041 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.475138 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.477945 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:36.477950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:36.477953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:36.477985 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:36.480525 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.481105 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.481110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.481112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.481253 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.481588 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.481592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.481594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.481933 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:36.482044 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.482338 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.482555 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:44:36.483724 4298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:44:36 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157076, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.252@KRBTEST.COM for oryx/127.25.137.252@KRBTEST.COM
I20260808 02:44:36.495417 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.484363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:42319 (local address 127.0.0.1:40708)
0808 02:44:36.484507 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:36.484521 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:36.484605 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:36.484816 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:36.484819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:36.485212 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:36.485355 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.485363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.486376 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.486381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.487242 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.487273 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.487947 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.487952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.488097 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.488586 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:36.488602 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:36.490468 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:36.493286 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.493290 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.493292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.493657 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.494054 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.494058 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.494060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.494217 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.494687 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:36.494690 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:36.494818 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 02:44:36.494934 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 02:44:36.495113 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 02:44:36.495497 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.484374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40708 (local address 127.25.137.252:42319)
0808 02:44:36.484520 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:36.484525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:36.484562 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:44:36.484672 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:36.484675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:36.484725 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:36.484801 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:36.485487 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.486229 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.487369 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.487830 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.487924 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.490637 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:36.490641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:36.490645 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:36.490674 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:36.493162 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.493764 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.493768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.493770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.493923 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.494321 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.494324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.494326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.494577 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:36.494684 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.495078 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.495249 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:44:36.500320 4298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: Bootstrap starting.
I20260808 02:44:36.503489 4298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:36.504520 4298 log.cc:824] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:36.507242 4298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: No bootstrap required, opened a new log
I20260808 02:44:36.514536 4298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } }
I20260808 02:44:36.514890 4298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:36.515008 4298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fdbdbfc708421c8fed116fef9abb70, State: Initialized, Role: FOLLOWER
I20260808 02:44:36.515525 4298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } }
I20260808 02:44:36.516436 4306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } } }
I20260808 02:44:36.516916 4306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:44:36.517825 4298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:44:36.528118 4322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:44:36.528281 4322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:44:36.531498 4050 master_runner.cc:428] Detected that this master 56fdbdbfc708421c8fed116fef9abb70 is joining an existing cluster
I20260808 02:44:36.531628 4050 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.137.252:42319
I20260808 02:44:36.553292 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.536386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60858 (local address 127.25.137.254:34061)
0808 02:44:36.536603 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:36.536609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:36.537216 (+ 607us) server_negotiation.cc:407] Connection header received
0808 02:44:36.537521 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:36.537527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:36.537609 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:36.537683 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:36.539379 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.540541 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.542679 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.543399 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.543499 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.546121 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:36.546128 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:36.546133 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:36.546173 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:36.549878 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.550734 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.550740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.550743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.550913 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.551424 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.551429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.551433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.551814 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:36.551964 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.552661 (+ 697us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.552902 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:44:36.553987 4203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.537523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.25.137.253:45953)
0808 02:44:36.537717 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:36.537723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:36.538397 (+ 674us) server_negotiation.cc:407] Connection header received
0808 02:44:36.538745 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:36.538750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:36.538838 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:36.538912 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:36.540482 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.541657 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.543876 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.544559 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.544651 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.547413 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:36.547420 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:36.547425 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:36.547470 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:36.550825 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.551597 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.551603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.551606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.551748 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.552261 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.552266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.552269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.552626 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:36.552741 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.553433 (+ 692us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.553654 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:44:36.554422 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.536423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:60858)
0808 02:44:36.537110 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:36.537130 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:36.537352 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:36.537907 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:36.537911 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:36.538780 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:36.539176 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.539190 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.540731 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.540736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.542496 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.542511 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.543791 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.543797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.543962 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.544688 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:36.544712 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:36.545898 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:36.550055 (+ 4157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.550060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.550064 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.550587 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.551065 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.551070 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.551074 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.551290 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.552067 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:36.552071 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:36.552316 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 02:44:36.553566 (+ 1250us) client_negotiation.cc:240] Negotiation successful
0808 02:44:36.553916 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":3,"thread_start_us":335,"threads_started":1}
I20260808 02:44:36.555812 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.537223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:45953 (local address 127.0.0.1:57970)
0808 02:44:36.538279 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:36.538295 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:36.538516 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:36.539155 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:36.539161 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:36.539899 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:36.540290 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.540313 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.541891 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.541896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.543687 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.543702 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.544970 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.544981 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.545239 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.545936 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:36.545960 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:36.547167 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:36.550968 (+ 3801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.550972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.550976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.551443 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.551890 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.551894 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.551897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.552120 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.552846 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:36.552851 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:36.553131 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 02:44:36.554242 (+ 1111us) client_negotiation.cc:240] Negotiation successful
0808 02:44:36.554582 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"thread_start_us":673,"threads_started":1}
I20260808 02:44:36.565550 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.538699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.25.137.252:42319)
0808 02:44:36.538881 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:36.538889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:36.548772 (+ 9883us) server_negotiation.cc:407] Connection header received
0808 02:44:36.549175 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:36.549180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:36.549251 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:36.549320 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:36.550760 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.551855 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.554382 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.555092 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.555207 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.558337 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:36.558350 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:36.558355 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:36.558391 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:36.561822 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.563281 (+ 1459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.563285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.563287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.563425 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.563816 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.563820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.563823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.564236 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:36.564449 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.564869 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.565178 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":50}
I20260808 02:44:36.565572 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.538238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:42319 (local address 127.0.0.1:40724)
0808 02:44:36.548672 (+ 10434us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:36.548691 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:36.549042 (+ 351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:36.549526 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:36.549530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:36.550235 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:36.550605 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.550620 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.552077 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.552083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.553988 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.554004 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.556229 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.556233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.556359 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.557043 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:36.557067 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:36.558162 (+ 1095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:36.562599 (+ 4437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.562606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.562608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.563108 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.563574 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.563579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.563583 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.563705 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.564491 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:36.564494 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:36.564619 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 02:44:36.564731 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 02:44:36.565024 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10267,"mutex_wait_us":5,"thread_start_us":10550,"threads_started":1}
Aug 08 02:44:36 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157073, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.254@KRBTEST.COM for oryx/127.25.137.252@KRBTEST.COM
I20260808 02:44:36.575157 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 3235
I20260808 02:44:36.576438 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.560616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.25.137.252:42319)
0808 02:44:36.561431 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:36.561437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:36.561459 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:44:36.561559 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:36.561564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:36.561695 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:36.561855 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:36.562621 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.564004 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.565763 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.566589 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.566710 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.570437 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:36.570442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:36.570445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:36.570474 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:36.573295 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.574047 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.574056 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.574060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.574212 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.574646 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.574651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.574655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.575047 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:36.575197 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.575711 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.575948 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":525,"threads_started":1}
I20260808 02:44:36.577059 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.560294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:42319 (local address 127.0.0.1:40732)
0808 02:44:36.560497 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:36.560518 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:36.560630 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:36.561873 (+ 1243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:36.561878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:36.562282 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:36.562451 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.562463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.564169 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.564175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.565574 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.565590 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.566759 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.566766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.567014 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.567646 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:36.567671 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:36.570209 (+ 2538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:36.573420 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.573426 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.573430 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.573909 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.574348 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.574353 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.574356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.574519 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.575186 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:36.575191 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:36.575370 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 02:44:36.575529 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 02:44:36.576615 (+ 1086us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 02:44:36.578469 3133 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.137.252:42319: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70"
member_type: NON_VOTER
last_known_addr {
host: "127.25.137.252"
port: 42319
}
attrs {
promote: true
}
}
dest_uuid: "62b3c63d1651463a974424f583236de8"
cas_config_opid_index: 5
I20260808 02:44:36.579744 3133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } }
I20260808 02:44:36.581655 4044 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 56fdbdbfc708421c8fed116fef9abb70 (127.25.137.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:36.582868 4266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:36.585428 4128 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 LEARNER]: Refusing update from remote peer 62b3c63d1651463a974424f583236de8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 02:44:36.586520 4266 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 FOLLOWER]: Refusing update from remote peer 62b3c63d1651463a974424f583236de8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 02:44:36.587031 4347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 02:44:36.587918 4347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
Aug 08 02:44:36 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.137.193@KRBTEST.COM
I20260808 02:44:36.597332 4044 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.137.252:42319
I20260808 02:44:36.598097 4050 master.cc:537] Master@127.25.137.252:42319 shutting down...
I20260808 02:44:36.601277 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.576237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35150 (local address 127.25.137.193:46793)
0808 02:44:36.577869 (+ 1632us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:36.577880 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:36.577917 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:44:36.577996 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:36.578007 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:36.578320 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:36.578533 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:36.580246 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.581949 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.584173 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.585149 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.586044 (+ 895us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.592463 (+ 6419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:36.592480 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:36.592499 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:36.592548 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:36.596694 (+ 4146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.597887 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.597898 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.597902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.598086 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.598841 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.598847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.598851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.599366 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:36.599664 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.600492 (+ 828us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.600793 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1424,"thread_start_us":1199,"threads_started":1}
I20260808 02:44:36.603711 3437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:36.603883 4348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:36.604364 4348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:36.605610 4128 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 56fdbdbfc708421c8fed116fef9abb70 (127.25.137.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:36.610334 4181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:36.610630 4181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:44:36.611126 4347 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8: attempting to promote NON_VOTER 90edc8737f924dfe82035aebcbc31b12 to VOTER
I20260808 02:44:36.611995 4347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } }
I20260808 02:44:36.614183 4128 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 LEARNER]: Refusing update from remote peer 62b3c63d1651463a974424f583236de8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 02:44:36.614847 4348 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 02:44:36.617301 4347 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 90edc8737f924dfe82035aebcbc31b12 (127.25.137.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:36.618098 4128 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 90edc8737f924dfe82035aebcbc31b12 (127.25.137.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:36.633215 4347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:36.633586 4347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:36.633545 4266 raft_consensus.cc:858] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } }
I20260808 02:44:36.634028 4347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:36.634343 4347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: This master's current role is: LEADER
W20260808 02:44:36.633759 4266 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } }
I20260808 02:44:36.640543 4181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:36.640796 4181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:44:36.650524 3111 consensus_peers.cc:487] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 -> Peer 56fdbdbfc708421c8fed116fef9abb70 (127.25.137.252:42319): Couldn't send request to peer 56fdbdbfc708421c8fed116fef9abb70. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 02:44:36.655157 4266 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 90edc8737f924dfe82035aebcbc31b12 (127.25.137.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } }
I20260808 02:44:36.659744 4369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } }
I20260808 02:44:36.660197 4369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 02:44:36.660274 4363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } }
I20260808 02:44:36.660641 4363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 02:44:36.660609 4306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62b3c63d1651463a974424f583236de8. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: NON_VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: true } } }
I20260808 02:44:36.660843 4306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 02:44:36.663797 4050 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260808 02:44:36.664256 4050 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260808 02:44:36.664448 4050 tablet_replica.cc:330] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: stopping tablet replica
I20260808 02:44:36.672786 4050 master.cc:552] Master@127.25.137.252:42319 shutdown complete.
I20260808 02:44:36.672878 4050 master_runner.cc:305] Clearing existing system tablet
I20260808 02:44:36.673271 4050 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:36.673377 4050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:36.676244 4050 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 02:44:36.677909 4371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:36.678277 4050 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:36.678551 4050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:44:36.678661 4050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/wal
uuid: "56fdbdbfc708421c8fed116fef9abb70"
format_stamp: "Formatted at 2026-08-08 02:44:36 on dist-test-slave-k5rr"
server_key: "c02b9c0464c569ba8be0098e1878a63d"
server_key_iv: "c600b4f89ca3a2e62c636d76b4cd9fde"
server_key_version: "encryptionkey@0"
I20260808 02:44:36.683318 4050 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:36.690748 4050 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 02:44:36.690904 4050 log.cc:1196] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 02:44:36.691395 4050 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: Deleting consensus metadata
I20260808 02:44:36.691671 4050 master_runner.cc:315] Copying system tablet from 127.25.137.254:34061
I20260808 02:44:36.698863 4050 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: tablet copy: Beginning tablet copy session from remote peer at address 127.25.137.254:34061
I20260808 02:44:36.720201 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.699775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.25.137.254:34061)
0808 02:44:36.700049 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:36.700056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:36.705661 (+ 5605us) server_negotiation.cc:407] Connection header received
0808 02:44:36.705848 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:36.705854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:36.705943 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:36.706060 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:36.707226 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.708357 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.710253 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.710919 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.711052 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.713450 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:36.713457 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:36.713462 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:36.713505 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:36.717046 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.717879 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.717885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.717889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.718052 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:36.718522 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:36.718527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:36.718531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:36.718920 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:36.719072 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:36.719587 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 02:44:36.719832 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260808 02:44:36.720240 4383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:36.699723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:60872)
0808 02:44:36.705513 (+ 5790us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:36.705534 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:36.705702 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:36.706098 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:36.706103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:36.706732 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:36.707041 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.707055 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.708517 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.708522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.710095 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:36.710110 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:36.711090 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:36.711096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:36.711286 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:36.712033 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:36.712060 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:36.713161 (+ 1101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:36.717177 (+ 4016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.717183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.717187 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.717744 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.718184 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:36.718189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:36.718193 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:36.718381 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:36.719060 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:36.719065 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:36.719305 (+ 240us) client_negotiation.cc:769] Sending connection context
0808 02:44:36.719442 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 02:44:36.719773 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5631,"thread_start_us":5479,"threads_started":1}
I20260808 02:44:36.721225 3162 tablet_copy_service.cc:140] P 62b3c63d1651463a974424f583236de8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 56fdbdbfc708421c8fed116fef9abb70 ({username='oryx', principal='oryx/127.25.137.252@KRBTEST.COM'} at 127.0.0.1:60872)
I20260808 02:44:36.721413 3162 tablet_copy_service.cc:161] P 62b3c63d1651463a974424f583236de8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 56fdbdbfc708421c8fed116fef9abb70 at {username='oryx', principal='oryx/127.25.137.252@KRBTEST.COM'} at 127.0.0.1:60872: session id = 56fdbdbfc708421c8fed116fef9abb70-00000000000000000000000000000000
I20260808 02:44:36.725206 3162 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 02:44:36.727190 4050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 02:44:36.734230 4050 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: tablet copy: Starting download of 0 data blocks...
I20260808 02:44:36.734560 4050 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: tablet copy: Starting download of 1 WAL segments...
I20260808 02:44:36.737232 4050 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 02:44:36.739774 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 3235
I20260808 02:44:36.743237 3162 tablet_copy_service.cc:342] P 62b3c63d1651463a974424f583236de8: Request end of tablet copy session 56fdbdbfc708421c8fed116fef9abb70-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.137.252@KRBTEST.COM'} at 127.0.0.1:60872
I20260808 02:44:36.743388 3162 tablet_copy_service.cc:433] P 62b3c63d1651463a974424f583236de8: ending tablet copy session 56fdbdbfc708421c8fed116fef9abb70-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 56fdbdbfc708421c8fed116fef9abb70
I20260808 02:44:36.758601 4050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:36.761076 4384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:36.767519 4385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:36.770149 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:46793
--local_ip_for_outbound_sockets=127.25.137.193
--tserver_master_addrs=127.25.137.254:34061
--webserver_port=40781
--webserver_interface=127.25.137.193
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab
--principal=oryx/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.137.254:34061,127.25.137.253:45953
--tserver_master_addrs=127.25.137.254:34061,127.25.137.253:45953,127.25.137.252:42319 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:36.770493 4387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:36.778018 4050 server_base.cc:1034] running on GCE node
I20260808 02:44:36.778326 4050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:36.781663 4050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:36.782884 4050 hybrid_clock.cc:648] HybridClock initialized: now 1786157076782825 us; error 43 us; skew 500 ppm
I20260808 02:44:36.783118 4050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:36 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157076, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:36.785372 4050 init.cc:377] Logged in from keytab as oryx/127.25.137.252@KRBTEST.COM (short username oryx)
I20260808 02:44:36.786825 4050 webserver.cc:460] Webserver started at http://127.25.137.252:35659/ using document root <none> and password file <none>
I20260808 02:44:36.787176 4050 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:36.787328 4050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:36.790441 4050 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 02:44:36.797415 4402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:36.797907 4050 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.006s
I20260808 02:44:36.798043 4050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/wal
uuid: "56fdbdbfc708421c8fed116fef9abb70"
format_stamp: "Formatted at 2026-08-08 02:44:36 on dist-test-slave-k5rr"
server_key: "c02b9c0464c569ba8be0098e1878a63d"
server_key_iv: "c600b4f89ca3a2e62c636d76b4cd9fde"
server_key_version: "encryptionkey@0"
I20260808 02:44:36.798257 4050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:36.822664 4050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:36.826817 4050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:36.851939 4050 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.252:42319
I20260808 02:44:36.852001 4479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.252:42319 every 8 connection(s)
I20260808 02:44:36.853000 4050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
I20260808 02:44:36.858318 4480 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:44:36.862326 4480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: Bootstrap starting.
I20260808 02:44:36.878940 4480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:44:36.879524 4480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: Bootstrap complete.
I20260808 02:44:36.880880 4480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } }
I20260808 02:44:36.881227 4480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 56fdbdbfc708421c8fed116fef9abb70, State: Initialized, Role: LEARNER
I20260808 02:44:36.881719 4480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } }
I20260808 02:44:36.882936 4480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:36.882725 4485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:36.883319 4485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:44:36.901073 4505 catalog_manager.cc:1259] Loaded cluster ID: 8e115c5b06734bc486ed5cec4912f3fa
I20260808 02:44:36.901182 4505 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: loading cluster ID for follower catalog manager: success
I20260808 02:44:36.904628 4505 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: acquiring CA information for follower catalog manager: success
I20260808 02:44:36.906983 4505 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:44:37.040113 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.034197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:42319 (local address 127.0.0.1:40748)
0808 02:44:37.034374 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:37.034395 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:37.034504 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:37.035376 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:37.035382 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:37.035413 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:37.035582 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.035591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.036877 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.036884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.038164 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.038180 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.039340 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.039347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.039550 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.039585 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:44:37.039688 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 02:44:37.039829 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 02:44:37.040210 4508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.034449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.25.137.252:42319)
0808 02:44:37.035045 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:37.035049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:37.035066 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:44:37.035130 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:37.035134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:37.035206 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:37.035317 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:37.035722 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.036720 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.038328 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.039154 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.039344 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.039532 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:37.039726 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 02:44:37.039866 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":306,"threads_started":1}
I20260808 02:44:37.041100 4442 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 02:44:37.041555 4485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62b3c63d1651463a974424f583236de8. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:37.041880 4485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: This master's current role is: LEARNER
W20260808 02:44:37.143281 4394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:37.143545 4394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:37.143680 4394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:37.151064 4394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:37.151175 4394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:37.151331 4394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:44:37.161630 4394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab
--principal=oryx/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:46793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=40781
--tserver_master_addrs=127.25.137.254:34061,127.25.137.253:45953,127.25.137.252:42319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:37.163008 4394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:37.164409 4394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:37.178185 4514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:37.178227 4517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:37.178262 4519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:37.179356 4394 server_base.cc:1034] running on GCE node
I20260808 02:44:37.179948 4394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:37.181375 4394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:37.182628 4394 hybrid_clock.cc:648] HybridClock initialized: now 1786157077182582 us; error 41 us; skew 500 ppm
I20260808 02:44:37.182941 4394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:37 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157077, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:37.186174 4394 init.cc:377] Logged in from keytab as oryx/127.25.137.193@KRBTEST.COM (short username oryx)
I20260808 02:44:37.187700 4394 webserver.cc:460] Webserver started at http://127.25.137.193:40781/ using document root <none> and password file <none>
I20260808 02:44:37.188356 4394 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:37.188490 4394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:37.193850 4394 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:37.196440 4529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:37.197760 4394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:44:37.197914 4394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "645595ba49ff4a60ac33aca9c703bc0a"
format_stamp: "Formatted at 2026-08-08 02:44:34 on dist-test-slave-k5rr"
server_key: "c9e1d497197ff33c085186c53c242f57"
server_key_iv: "6f8f7e52fdbb279e40d8d9a98d045bb8"
server_key_version: "encryptionkey@0"
I20260808 02:44:37.198945 4394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:37.219194 4394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:37.223223 4394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:37.223475 4394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:37.224416 4394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:37.225831 4394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:37.225919 4394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:37.226002 4394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:37.226058 4394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:37.262435 4394 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:46793
I20260808 02:44:37.262516 4708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:46793 every 8 connection(s)
I20260808 02:44:37.263896 4394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:44:37.272257 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 4394
I20260808 02:44:37.272599 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 3459
Aug 08 02:44:37 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157077, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.252@KRBTEST.COM
Aug 08 02:44:37 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157077, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
Aug 08 02:44:37 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157077, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.253@KRBTEST.COM
I20260808 02:44:37.291730 4508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.267078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:35015 (local address 127.25.137.252:42319)
0808 02:44:37.267260 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:37.267264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:37.268534 (+ 1270us) server_negotiation.cc:407] Connection header received
0808 02:44:37.271391 (+ 2857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:37.271398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:37.271501 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:37.271601 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:37.274249 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.275380 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.277895 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.278512 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.278606 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.283896 (+ 5290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:37.283904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:37.283909 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:37.283950 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:37.287776 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.289356 (+ 1580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.289361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.289364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.289599 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.290035 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.290040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.290042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.290380 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:37.290509 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:37.291093 (+ 584us) server_negotiation.cc:299] Negotiation successful
0808 02:44:37.291376 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 02:44:37 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.137.194@KRBTEST.COM
I20260808 02:44:37.292955 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.268838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.193:55059)
0808 02:44:37.269527 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:37.269544 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:37.271095 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:37.271913 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:37.271923 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:37.272930 (+ 1007us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:37.274046 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.274063 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.275619 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.275625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.278072 (+ 2447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.278085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.279377 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.279384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.279558 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.280326 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:37.280346 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:37.283900 (+ 3554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:37.288052 (+ 4152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.288061 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.288079 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.288701 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.289209 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.289213 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.289217 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.289436 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.290178 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:37.290186 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:37.290612 (+ 426us) client_negotiation.cc:769] Sending connection context
0808 02:44:37.290964 (+ 352us) client_negotiation.cc:240] Negotiation successful
0808 02:44:37.291470 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":216,"spinlock_wait_cycles":31488,"thread_start_us":318,"threads_started":1}
I20260808 02:44:37.293606 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.269665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:45953 (local address 127.25.137.193:42159)
0808 02:44:37.270308 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:37.270325 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:37.271103 (+ 778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:37.272239 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:37.272243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:37.272894 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:37.274031 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.274058 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.276245 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.276251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.278068 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.278085 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.279189 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.279196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.279430 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.280187 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:37.280214 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:37.286238 (+ 6024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:37.290379 (+ 4141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.290384 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.290388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.291006 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.291479 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.291499 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.291503 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.291670 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.292574 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:37.292579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:37.292729 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 02:44:37.292873 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 02:44:37.293164 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":175,"spinlock_wait_cycles":38016,"thread_start_us":297,"threads_started":1}
I20260808 02:44:37.293642 4720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.268970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:55059 (local address 127.25.137.254:34061)
0808 02:44:37.269619 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:37.269624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:37.269645 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:44:37.271405 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:37.271411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:37.271500 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:37.271581 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:37.274273 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.275412 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.278232 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.278874 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.278969 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.284114 (+ 5145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:37.284121 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:37.284125 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:37.284158 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:37.287903 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.288868 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.288873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.288877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.289076 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.289584 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.289589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.289591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.289958 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:37.290104 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:37.293065 (+ 2961us) server_negotiation.cc:299] Negotiation successful
0808 02:44:37.293310 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":412,"threads_started":1}
I20260808 02:44:37.293774 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.267565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:42319 (local address 127.25.137.193:35015)
0808 02:44:37.268351 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:37.268412 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:37.271091 (+ 2679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:37.271916 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:37.271924 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:37.272877 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:37.274031 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.274058 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.275611 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.275616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.277697 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.277712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.278961 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.278969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.279301 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.280072 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:37.280105 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:37.283565 (+ 3460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:37.288708 (+ 5143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.288712 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.288716 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.289207 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.289740 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.289745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.289748 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.289901 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.290588 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:37.290592 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:37.290786 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 02:44:37.291948 (+ 1162us) client_negotiation.cc:240] Negotiation successful
0808 02:44:37.292242 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":221,"spinlock_wait_cycles":1920,"thread_start_us":299,"threads_started":1}
I20260808 02:44:37.294592 4724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.269739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:42159 (local address 127.25.137.253:45953)
0808 02:44:37.270468 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:37.270474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:37.270502 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:44:37.271815 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:37.271821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:37.271921 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:37.272020 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:37.274235 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.275420 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.278257 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.278958 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.279773 (+ 815us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.286617 (+ 6844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:37.286625 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:37.286630 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:37.286671 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:37.290209 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.291175 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.291180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.291184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.291349 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.292070 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.292075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.292078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.292431 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:37.293902 (+ 1471us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:37.294051 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 02:44:37.294266 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":422,"threads_started":1}
I20260808 02:44:37.295120 4711 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34061
I20260808 02:44:37.295313 4712 heartbeater.cc:347] Connected to a master server at 127.25.137.252:42319
I20260808 02:44:37.295480 4712 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:37.295487 4711 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:37.296270 4713 heartbeater.cc:347] Connected to a master server at 127.25.137.253:45953
I20260808 02:44:37.296429 4713 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:37.296434 4711 heartbeater.cc:511] Master 127.25.137.254:34061 requested a full tablet report, sending...
I20260808 02:44:37.296559 4712 heartbeater.cc:511] Master 127.25.137.252:42319 requested a full tablet report, sending...
I20260808 02:44:37.296876 4713 heartbeater.cc:511] Master 127.25.137.253:45953 requested a full tablet report, sending...
I20260808 02:44:37.298427 3133 ts_manager.cc:194] Re-registered known tserver with Master: 645595ba49ff4a60ac33aca9c703bc0a (127.25.137.193:46793)
I20260808 02:44:37.299000 4430 ts_manager.cc:194] Registered new tserver with Master: 645595ba49ff4a60ac33aca9c703bc0a (127.25.137.193:46793)
I20260808 02:44:37.299300 3133 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:55059
I20260808 02:44:37.300551 4116 ts_manager.cc:194] Registered new tserver with Master: 645595ba49ff4a60ac33aca9c703bc0a (127.25.137.193:46793)
I20260808 02:44:37.303267 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.273984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.25.137.194:44887)
0808 02:44:37.280573 (+ 6589us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:37.280581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:37.280613 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:44:37.280685 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:37.280693 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:37.280968 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:37.283153 (+ 2185us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:37.283427 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.284887 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.286853 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.287716 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.288458 (+ 742us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.295282 (+ 6824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:37.295297 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:37.295312 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:37.295359 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:37.300346 (+ 4987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.301332 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.301340 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.301343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.301505 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.301878 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.301881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.301883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.302221 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:37.302443 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:37.302663 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 02:44:37.302899 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6069,"thread_start_us":6244,"threads_started":1}
I20260808 02:44:37.304363 3654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:37.387518 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 3459
I20260808 02:44:37.413215 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:44887
--local_ip_for_outbound_sockets=127.25.137.194
--tserver_master_addrs=127.25.137.254:34061
--webserver_port=41713
--webserver_interface=127.25.137.194
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab
--principal=oryx/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.137.254:34061,127.25.137.253:45953
--tserver_master_addrs=127.25.137.254:34061,127.25.137.253:45953,127.25.137.252:42319 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:44:37.608214 4347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: NON_VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: true } } }
I20260808 02:44:37.608635 4347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:37.610209 4730 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8: attempting to promote NON_VOTER 56fdbdbfc708421c8fed116fef9abb70 to VOTER
I20260808 02:44:37.610786 4730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } }
I20260808 02:44:37.612643 4442 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 LEARNER]: Refusing update from remote peer 62b3c63d1651463a974424f583236de8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 02:44:37.612643 4128 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 FOLLOWER]: Refusing update from remote peer 62b3c63d1651463a974424f583236de8: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 02:44:37.613332 4730 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 02:44:37.613709 4347 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 02:44:37.616226 4730 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 56fdbdbfc708421c8fed116fef9abb70 (127.25.137.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:37.616966 4128 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 56fdbdbfc708421c8fed116fef9abb70 (127.25.137.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:37.617954 4442 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 56fdbdbfc708421c8fed116fef9abb70 (127.25.137.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:37.620148 4730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:37.620383 4730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:37.620657 4730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:37.620853 4730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:37.621165 4730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:37.621407 4730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:37.625501 4736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:37.625905 4736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:44:37.633096 4740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62b3c63d1651463a974424f583236de8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:37.633435 4740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:44:37.751427 4728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:37.751757 4728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:37.751914 4728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:37.759127 4728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:37.759234 4728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:37.759392 4728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:44:37.768934 4728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab
--principal=oryx/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:44887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=41713
--tserver_master_addrs=127.25.137.254:34061,127.25.137.253:45953,127.25.137.252:42319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:37.770174 4728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:37.771463 4728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:37.784087 4750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:37.784235 4753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:37.784606 4751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:37.785557 4728 server_base.cc:1034] running on GCE node
I20260808 02:44:37.786326 4728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:37.787317 4728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:37.788560 4728 hybrid_clock.cc:648] HybridClock initialized: now 1786157077788495 us; error 46 us; skew 500 ppm
I20260808 02:44:37.788930 4728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:37 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157077, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:37.792117 4728 init.cc:377] Logged in from keytab as oryx/127.25.137.194@KRBTEST.COM (short username oryx)
I20260808 02:44:37.793515 4728 webserver.cc:460] Webserver started at http://127.25.137.194:41713/ using document root <none> and password file <none>
I20260808 02:44:37.794265 4728 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:37.794399 4728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:37.800006 4728 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:44:37.802870 4768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:37.804198 4728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:44:37.804347 4728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "b2794284317c4a899881ef7bc6ce223f"
format_stamp: "Formatted at 2026-08-08 02:44:34 on dist-test-slave-k5rr"
server_key: "368cc69bee90d17faed63a2e0b166b67"
server_key_iv: "3d1ea8b50e02d8887057333b67f494d3"
server_key_version: "encryptionkey@0"
I20260808 02:44:37.805354 4728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:37.825320 4728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:37.829137 4728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:37.829399 4728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:37.830399 4728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:37.831779 4728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:37.831873 4728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:37.831984 4728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:37.832059 4728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:37.869112 4728 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:44887
I20260808 02:44:37.869166 4943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:44887 every 8 connection(s)
I20260808 02:44:37.870752 4728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:44:37.877848 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 4728
I20260808 02:44:37.878093 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 3668
Aug 08 02:44:37 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157077, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
Aug 08 02:44:37 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157077, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.253@KRBTEST.COM
Aug 08 02:44:37 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157077, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.252@KRBTEST.COM
I20260808 02:44:37.902105 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.873691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:40715 (local address 127.25.137.253:45953)
0808 02:44:37.874190 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:37.874194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:37.875108 (+ 914us) server_negotiation.cc:407] Connection header received
0808 02:44:37.878288 (+ 3180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:37.878295 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:37.878385 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:37.878467 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:37.881141 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.882505 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.885053 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.885763 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.885871 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.894913 (+ 9042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:37.894921 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:37.894926 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:37.894975 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:37.898510 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.899367 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.899373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.899376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.899537 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.900052 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.900058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.900062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.900482 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:37.900648 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:37.901451 (+ 803us) server_negotiation.cc:299] Negotiation successful
0808 02:44:37.901689 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":226,"threads_started":1}
I20260808 02:44:37.902159 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.876201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:49273 (local address 127.25.137.254:34061)
0808 02:44:37.877383 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:37.877390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:37.877433 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:44:37.878200 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:37.878207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:37.878328 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:37.878445 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:37.881549 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.884450 (+ 2901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.886346 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.887000 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.887930 (+ 930us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.891560 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:37.891567 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:37.891572 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:37.891626 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:37.896569 (+ 4943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.899037 (+ 2468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.899042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.899044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.899179 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.899912 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.899917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.899984 (+ 67us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.900341 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:37.900487 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:37.901411 (+ 924us) server_negotiation.cc:299] Negotiation successful
0808 02:44:37.901642 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1055,"thread_start_us":930,"threads_started":1}
I20260808 02:44:37.903427 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.875371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:42319 (local address 127.25.137.194:39115)
0808 02:44:37.875996 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:37.876021 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:37.877765 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:37.878949 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:37.878954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:37.879608 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:37.880875 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.880903 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.887193 (+ 6290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.887199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.888788 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.888805 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.891408 (+ 2603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.891417 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.891613 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.892167 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:37.892192 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:37.895242 (+ 3050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:37.899529 (+ 4287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.899534 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.899538 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.900163 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.900693 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.900698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.900702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.900881 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.901596 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:37.901602 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:37.901779 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 02:44:37.901920 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 02:44:37.902198 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":19,"spinlock_wait_cycles":26368,"thread_start_us":269,"threads_started":1}
I20260808 02:44:37.904215 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.874223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:45953 (local address 127.25.137.194:40715)
0808 02:44:37.874947 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:37.874993 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:37.878129 (+ 3136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:37.878758 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:37.878763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:37.880415 (+ 1652us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:37.880875 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.880903 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.882753 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.882759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.884827 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.884843 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.886305 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.886313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.886688 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.890498 (+ 3810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:37.890530 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:37.894623 (+ 4093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:37.898691 (+ 4068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.898696 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.898700 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.899195 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.899699 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.899704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.899708 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.899898 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.900727 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:37.900732 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:37.900967 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 02:44:37.902439 (+ 1472us) client_negotiation.cc:240] Negotiation successful
0808 02:44:37.902740 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":23,"spinlock_wait_cycles":28416,"thread_start_us":267,"threads_started":1}
I20260808 02:44:37.904258 4958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.875540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:39115 (local address 127.25.137.252:42319)
0808 02:44:37.876973 (+ 1433us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:37.876980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:37.877020 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:44:37.878657 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:37.878665 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:37.878767 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:37.879797 (+ 1030us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:37.881807 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.884262 (+ 2455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.889059 (+ 4797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.891098 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.891939 (+ 841us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.895489 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:37.895496 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:37.895501 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:37.895544 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:37.899378 (+ 3834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.900367 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.900372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.900376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.900523 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.901074 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.901079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.901083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.901453 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:37.903435 (+ 1982us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:37.903578 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 02:44:37.903820 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1278,"thread_start_us":1142,"threads_started":1}
I20260808 02:44:37.905550 4949 heartbeater.cc:347] Connected to a master server at 127.25.137.253:45953
I20260808 02:44:37.905617 4946 heartbeater.cc:347] Connected to a master server at 127.25.137.252:42319
I20260808 02:44:37.905900 4946 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:37.905906 4949 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:37.906474 4959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.876073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.194:49273)
0808 02:44:37.876710 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:37.876728 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:37.877765 (+ 1037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:37.878745 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:37.878757 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:37.879958 (+ 1201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:37.881360 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.881374 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.884604 (+ 3230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.884611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.886139 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:37.886154 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.887352 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.887358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:37.887586 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.888183 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:37.888216 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:37.891265 (+ 3049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:37.896790 (+ 5525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.896801 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.896819 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.898801 (+ 1982us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.899305 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:37.899310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:37.899313 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:37.899564 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:37.900519 (+ 955us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:37.900528 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:37.900967 (+ 439us) client_negotiation.cc:769] Sending connection context
0808 02:44:37.901289 (+ 322us) client_negotiation.cc:240] Negotiation successful
0808 02:44:37.901827 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":6,"spinlock_wait_cycles":5120,"thread_start_us":353,"threads_started":1}
Aug 08 02:44:37 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.137.195@KRBTEST.COM
I20260808 02:44:37.906994 4946 heartbeater.cc:511] Master 127.25.137.252:42319 requested a full tablet report, sending...
I20260808 02:44:37.907359 4949 heartbeater.cc:511] Master 127.25.137.253:45953 requested a full tablet report, sending...
I20260808 02:44:37.907778 4944 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34061
I20260808 02:44:37.907920 4944 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:37.908372 4944 heartbeater.cc:511] Master 127.25.137.254:34061 requested a full tablet report, sending...
I20260808 02:44:37.909305 4116 ts_manager.cc:194] Registered new tserver with Master: b2794284317c4a899881ef7bc6ce223f (127.25.137.194:44887)
I20260808 02:44:37.909493 3133 ts_manager.cc:194] Re-registered known tserver with Master: b2794284317c4a899881ef7bc6ce223f (127.25.137.194:44887)
I20260808 02:44:37.910669 3133 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:49273
I20260808 02:44:37.910732 4430 ts_manager.cc:194] Registered new tserver with Master: b2794284317c4a899881ef7bc6ce223f (127.25.137.194:44887)
I20260808 02:44:37.915004 4964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:37.879408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.25.137.195:33541)
0808 02:44:37.892444 (+ 13036us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:37.892451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:37.892475 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:44:37.892530 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:37.892535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:37.892726 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:37.892896 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:37.895077 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.896628 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.898194 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:37.899542 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:37.899669 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:37.908530 (+ 8861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:37.908544 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:37.908556 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:37.908591 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:37.911975 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.912644 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.912652 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.912656 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.912789 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:37.913235 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:37.913240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:37.913244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:37.913569 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:37.913866 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:37.914174 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 02:44:37.914459 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12888,"thread_start_us":12710,"threads_started":1}
I20260808 02:44:37.916217 3869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:37.998831 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 3668
I20260808 02:44:38.020037 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:33541
--local_ip_for_outbound_sockets=127.25.137.195
--tserver_master_addrs=127.25.137.254:34061
--webserver_port=38683
--webserver_interface=127.25.137.195
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab
--principal=oryx/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.137.254:34061,127.25.137.253:45953
--tserver_master_addrs=127.25.137.254:34061,127.25.137.253:45953,127.25.137.252:42319 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:44:38.301994 4711 heartbeater.cc:503] Master 127.25.137.254:34061 was elected leader, sending a full tablet report...
W20260808 02:44:38.349577 4967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:38.349846 4967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:38.349949 4967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:38.357617 4967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:38.357754 4967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:38.357887 4967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:44:38.368103 4967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab
--principal=oryx/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:33541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=38683
--tserver_master_addrs=127.25.137.254:34061,127.25.137.253:45953,127.25.137.252:42319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:38.369494 4967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:38.370792 4967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:38.384864 4978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:38.384900 4982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:38.384788 4980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:38.385988 4967 server_base.cc:1034] running on GCE node
I20260808 02:44:38.386698 4967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:38.387723 4967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:38.389038 4967 hybrid_clock.cc:648] HybridClock initialized: now 1786157078388957 us; error 40 us; skew 500 ppm
I20260808 02:44:38.389410 4967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:38 dist-test-slave-k5rr krb5kdc[3068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157078, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:38.392699 4967 init.cc:377] Logged in from keytab as oryx/127.25.137.195@KRBTEST.COM (short username oryx)
I20260808 02:44:38.394207 4967 webserver.cc:460] Webserver started at http://127.25.137.195:38683/ using document root <none> and password file <none>
I20260808 02:44:38.394876 4967 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:38.395005 4967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:38.400527 4967 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 02:44:38.403371 4992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:38.404860 4967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:44:38.405061 4967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "cee9bde7a81745e09b5449e36a323e39"
format_stamp: "Formatted at 2026-08-08 02:44:35 on dist-test-slave-k5rr"
server_key: "782f94352a9cdf1cb34ab441d05cae2b"
server_key_iv: "a804700ac3890e0d56ec126450f8bfc4"
server_key_version: "encryptionkey@0"
I20260808 02:44:38.406050 4967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:38.424722 4967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:38.428421 4967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:38.428674 4967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:38.429667 4967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:38.430982 4967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:38.431077 4967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:38.431185 4967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:38.431255 4967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:38.461866 4967 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:33541
I20260808 02:44:38.461946 5161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:33541 every 8 connection(s)
I20260808 02:44:38.463323 4967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:44:38.465268 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 4967
Aug 08 02:44:38 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157078, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.252@KRBTEST.COM
I20260808 02:44:38.492400 5177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.467109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:42319 (local address 127.25.137.195:38967)
0808 02:44:38.468098 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:38.468146 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:38.470799 (+ 2653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:38.471999 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:38.472013 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:38.472837 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:38.473761 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.473787 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.475435 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.475440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.477786 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.477804 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.479155 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.479162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.479493 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.480298 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:38.480336 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:38.483827 (+ 3491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:38.487877 (+ 4050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:38.487888 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:38.487905 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:38.488601 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:38.489115 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:38.489120 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:38.489123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:38.489362 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:38.490007 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:38.490014 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:38.490368 (+ 354us) client_negotiation.cc:769] Sending connection context
0808 02:44:38.490677 (+ 309us) client_negotiation.cc:240] Negotiation successful
0808 02:44:38.491275 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":446,"threads_started":1}
I20260808 02:44:38.493654 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.466408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:38967 (local address 127.25.137.252:42319)
0808 02:44:38.466884 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.466890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.468285 (+ 1395us) server_negotiation.cc:407] Connection header received
0808 02:44:38.471180 (+ 2895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.471188 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.471304 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.471406 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:38.473993 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.475201 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.477976 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.478672 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.478770 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.484097 (+ 5327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:38.484104 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:38.484109 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:38.484154 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:38.487663 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.488754 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.488761 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.488765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.488925 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.489485 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.489491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.489494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.489879 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:38.490018 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.493026 (+ 3008us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.493269 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":236,"threads_started":1}
Aug 08 02:44:38 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157078, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.253@KRBTEST.COM
I20260808 02:44:38.496039 5164 heartbeater.cc:347] Connected to a master server at 127.25.137.252:42319
I20260808 02:44:38.496341 5164 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:38.497129 5164 heartbeater.cc:511] Master 127.25.137.252:42319 requested a full tablet report, sending...
I20260808 02:44:38.498952 4430 ts_manager.cc:194] Registered new tserver with Master: cee9bde7a81745e09b5449e36a323e39 (127.25.137.195:33541)
Aug 08 02:44:38 dist-test-slave-k5rr krb5kdc[3068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157078, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:38.507670 5180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.468875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:38931 (local address 127.25.137.253:45953)
0808 02:44:38.484429 (+ 15554us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.484435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.484456 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:44:38.484515 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.484520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.484602 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.484713 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:38.486653 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.487878 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.490486 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.491117 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.491244 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.499660 (+ 8416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:38.499666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:38.499671 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:38.499714 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:38.504220 (+ 4506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.505477 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.505483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.505486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.505658 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.506063 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.506069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.506072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.506470 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:38.506641 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.507044 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.507263 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":15412,"thread_start_us":15949,"threads_started":1}
I20260808 02:44:38.507692 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.468734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:45953 (local address 127.25.137.195:38931)
0808 02:44:38.482099 (+ 13365us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:38.482130 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:38.482246 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:38.484757 (+ 2511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:38.484763 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:38.485857 (+ 1094us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:38.486462 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.486475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.488620 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.488625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.490323 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.490341 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.492480 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.492491 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.492693 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.493418 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:38.493443 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:38.499412 (+ 5969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:38.504693 (+ 5281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:38.504699 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:38.504704 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:38.505300 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:38.505780 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:38.505785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:38.505789 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:38.505943 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:38.506598 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:38.506604 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:38.506778 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 02:44:38.506928 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 02:44:38.507231 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13246,"mutex_wait_us":1,"thread_start_us":12520,"threads_started":1}
I20260808 02:44:38.508890 5166 heartbeater.cc:347] Connected to a master server at 127.25.137.253:45953
I20260808 02:44:38.509065 5166 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:38.509574 5166 heartbeater.cc:511] Master 127.25.137.253:45953 requested a full tablet report, sending...
I20260808 02:44:38.510979 4116 ts_manager.cc:194] Registered new tserver with Master: cee9bde7a81745e09b5449e36a323e39 (127.25.137.195:33541)
I20260808 02:44:38.514055 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.481850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:42113 (local address 127.25.137.254:34061)
0808 02:44:38.487393 (+ 5543us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.487399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.487418 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:44:38.487486 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.487492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.487571 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.487684 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:38.490985 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.492193 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.497491 (+ 5298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.498326 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.498459 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.503800 (+ 5341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:38.503807 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:38.503812 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:38.503855 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:38.507465 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.511468 (+ 4003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.511473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.511481 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.511677 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.512576 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.512580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.512582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.512845 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:38.512983 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.513459 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.513689 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5452,"thread_start_us":5042,"threads_started":1}
I20260808 02:44:38.514092 5185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.481757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.195:42113)
0808 02:44:38.485486 (+ 3729us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:38.485510 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:38.485624 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:38.488042 (+ 2418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:38.488047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:38.488687 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:38.489137 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.489147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.492360 (+ 3213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.492366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.493941 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.493956 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.498997 (+ 5041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.499007 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.499265 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.500515 (+ 1250us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:38.500538 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:38.503583 (+ 3045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:38.507627 (+ 4044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:38.507631 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:38.507635 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:38.510150 (+ 2515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:38.512260 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:38.512265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:38.512270 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:38.512463 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:38.512972 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:38.512976 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:38.513168 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 02:44:38.513326 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 02:44:38.513623 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3586,"thread_start_us":4825,"threads_started":1}
I20260808 02:44:38.515136 5162 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34061
I20260808 02:44:38.515285 5162 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:38.515720 5162 heartbeater.cc:511] Master 127.25.137.254:34061 requested a full tablet report, sending...
I20260808 02:44:38.516768 3133 ts_manager.cc:194] Re-registered known tserver with Master: cee9bde7a81745e09b5449e36a323e39 (127.25.137.195:33541)
I20260808 02:44:38.517916 3133 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:42113
I20260808 02:44:38.518003 5187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.470140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60888 (local address 127.25.137.254:34061)
0808 02:44:38.487105 (+ 16965us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.487111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.490511 (+ 3400us) server_negotiation.cc:407] Connection header received
0808 02:44:38.490581 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.490587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.490669 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.490771 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:38.497476 (+ 6705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.498737 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.501960 (+ 3223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.502666 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.502797 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.508556 (+ 5759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:38.508564 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:38.508569 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:38.508608 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:38.512342 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.514428 (+ 2086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.514434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.514437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.514579 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.515473 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.515478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.515481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.515814 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:38.516063 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.517078 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.517587 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":268,"server-negotiator.queue_time_us":16843,"thread_start_us":17065,"threads_started":1}
I20260808 02:44:38.525345 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.502941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.25.137.252:42319)
0808 02:44:38.503111 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.503116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.506761 (+ 3645us) server_negotiation.cc:407] Connection header received
0808 02:44:38.507998 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.508003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.508077 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.508138 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:38.511069 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.512046 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.514185 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.514793 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.514918 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.518939 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:38.518946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:38.518950 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:38.518990 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:38.522221 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.522912 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.522918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.522922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.523089 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.523571 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.523576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.523614 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.523992 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:38.524259 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.524774 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.525036 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:44:38.522899 3133 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:44:38.531564 3133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:44:38.532771 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.488514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.25.137.253:45953)
0808 02:44:38.489793 (+ 1279us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.489799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.505434 (+ 15635us) server_negotiation.cc:407] Connection header received
0808 02:44:38.505664 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.505669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.506103 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.506210 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:38.510626 (+ 4416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.512085 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.514679 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.515410 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.515513 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.520077 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:38.520084 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:38.520089 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:38.520127 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:38.530008 (+ 9881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.530811 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.530815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.530817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.530994 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.531385 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.531388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.531390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.531657 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:38.531890 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.532295 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.532474 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1702,"server-negotiator.queue_time_us":1157,"thread_start_us":1031,"threads_started":1}
I20260808 02:44:38.583161 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.555651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50836 (local address 127.25.137.194:44887)
0808 02:44:38.575122 (+ 19471us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.575128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.575152 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:44:38.575242 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.575249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.575548 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.575778 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:38.576538 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.577867 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.580534 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.582013 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.582187 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.582356 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.582738 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.582926 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19352,"thread_start_us":19018,"threads_started":1}
I20260808 02:44:38.583177 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.555333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:44887 (local address 127.0.0.1:50836)
0808 02:44:38.570431 (+ 15098us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:38.570455 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:38.570616 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:38.575798 (+ 5182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:38.575803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:38.575836 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:38.576328 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.576341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.578014 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.578019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.580378 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.580391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.582288 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.582294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.582516 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.582552 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:44:38.582648 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 02:44:38.582772 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14942,"mutex_wait_us":22,"thread_start_us":15386,"threads_started":1}
I20260808 02:44:38.585757 5231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.557354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:33541 (local address 127.0.0.1:57146)
0808 02:44:38.576339 (+ 18985us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:38.576361 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:38.576478 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:38.576987 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:38.576992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:38.577055 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:38.577399 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.577411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.580429 (+ 3018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.580433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.582585 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.582598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.584849 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.584856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.585155 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.585189 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:44:38.585276 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 02:44:38.585396 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18861,"mutex_wait_us":74,"thread_start_us":18667,"threads_started":1}
I20260808 02:44:38.586086 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.557721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57146 (local address 127.25.137.195:33541)
0808 02:44:38.573620 (+ 15899us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.573631 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.576453 (+ 2822us) server_negotiation.cc:407] Connection header received
0808 02:44:38.576618 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.576626 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.576805 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.576968 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:38.577607 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.579305 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.582725 (+ 3420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.584663 (+ 1938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.584800 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.584912 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.585356 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.585512 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15695,"thread_start_us":15513,"threads_started":1}
I20260808 02:44:38.587244 4843 tablet_service.cc:1467] Processing CreateTablet for tablet c8cb494a616e49a9af452e422275322c (DEFAULT_TABLE table=test-table [id=dbff5a0ac9e54d68994099ae07156947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:38.587844 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.555747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46793 (local address 127.0.0.1:35160)
0808 02:44:38.576293 (+ 20546us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:38.576325 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:38.577236 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:38.578510 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:38.578513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:38.578532 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:38.578848 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.578854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.581128 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.581132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.585429 (+ 4297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.585443 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.587189 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.587193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.587322 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.587342 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:44:38.587443 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 02:44:38.587529 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20385,"mutex_wait_us":56,"thread_start_us":18540,"threads_started":1}
I20260808 02:44:38.588454 5229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.556045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35160 (local address 127.25.137.193:46793)
0808 02:44:38.574113 (+ 18068us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.574122 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.576436 (+ 2314us) server_negotiation.cc:407] Connection header received
0808 02:44:38.577407 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.577413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.577578 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.577773 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:38.579039 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.580293 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.585614 (+ 5321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.586826 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.586936 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.587044 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.588067 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.588228 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17892,"thread_start_us":17758,"threads_started":1}
I20260808 02:44:38.588519 4843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8cb494a616e49a9af452e422275322c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:38.595289 5068 tablet_service.cc:1467] Processing CreateTablet for tablet c8cb494a616e49a9af452e422275322c (DEFAULT_TABLE table=test-table [id=dbff5a0ac9e54d68994099ae07156947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:38.595304 4605 tablet_service.cc:1467] Processing CreateTablet for tablet c8cb494a616e49a9af452e422275322c (DEFAULT_TABLE table=test-table [id=dbff5a0ac9e54d68994099ae07156947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:38.596488 5068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8cb494a616e49a9af452e422275322c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:38.596583 4605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8cb494a616e49a9af452e422275322c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:38.603567 5233 tablet_bootstrap.cc:492] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: Bootstrap starting.
I20260808 02:44:38.607231 5233 tablet_bootstrap.cc:654] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:38.608525 5233 log.cc:824] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:38.610059 5234 tablet_bootstrap.cc:492] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: Bootstrap starting.
I20260808 02:44:38.612047 5236 tablet_bootstrap.cc:492] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: Bootstrap starting.
I20260808 02:44:38.612834 5233 tablet_bootstrap.cc:492] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: No bootstrap required, opened a new log
I20260808 02:44:38.613137 5233 ts_tablet_manager.cc:1397] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260808 02:44:38.613459 5234 tablet_bootstrap.cc:654] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:38.614389 5234 log.cc:824] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:38.615453 5236 tablet_bootstrap.cc:654] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:38.616627 5236 log.cc:824] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:38.618062 5234 tablet_bootstrap.cc:492] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: No bootstrap required, opened a new log
I20260808 02:44:38.618310 5234 ts_tablet_manager.cc:1397] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 02:44:38.619650 5236 tablet_bootstrap.cc:492] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: No bootstrap required, opened a new log
I20260808 02:44:38.619920 5236 ts_tablet_manager.cc:1397] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 02:44:38.623309 5233 raft_consensus.cc:348] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } } peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } }
I20260808 02:44:38.623755 5233 raft_consensus.cc:374] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:38.623847 5233 raft_consensus.cc:729] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2794284317c4a899881ef7bc6ce223f, State: Initialized, Role: FOLLOWER
I20260808 02:44:38.624552 5233 consensus_queue.cc:260] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } } peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } }
I20260808 02:44:38.627313 5234 raft_consensus.cc:348] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } } peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } }
I20260808 02:44:38.627707 5234 raft_consensus.cc:374] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:38.627805 5234 raft_consensus.cc:729] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cee9bde7a81745e09b5449e36a323e39, State: Initialized, Role: FOLLOWER
I20260808 02:44:38.628357 5234 consensus_queue.cc:260] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } } peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } }
I20260808 02:44:38.630348 5236 raft_consensus.cc:348] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } } peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } }
I20260808 02:44:38.630797 5236 raft_consensus.cc:374] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:38.630900 5236 raft_consensus.cc:729] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 645595ba49ff4a60ac33aca9c703bc0a, State: Initialized, Role: FOLLOWER
I20260808 02:44:38.631601 5236 consensus_queue.cc:260] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } } peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } }
I20260808 02:44:38.632740 4944 heartbeater.cc:503] Master 127.25.137.254:34061 was elected leader, sending a full tablet report...
I20260808 02:44:38.633478 5233 ts_tablet_manager.cc:1428] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: Time spent starting tablet: real 0.020s user 0.011s sys 0.009s
I20260808 02:44:38.636404 5162 heartbeater.cc:503] Master 127.25.137.254:34061 was elected leader, sending a full tablet report...
I20260808 02:44:38.636953 5234 ts_tablet_manager.cc:1428] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: Time spent starting tablet: real 0.019s user 0.010s sys 0.008s
I20260808 02:44:38.643646 5236 ts_tablet_manager.cc:1428] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: Time spent starting tablet: real 0.024s user 0.013s sys 0.004s
W20260808 02:44:38.716784 5168 tablet.cc:2369] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:44:38.767728 4714 tablet.cc:2369] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:44:38.783988 5241 raft_consensus.cc:482] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:38.784293 5241 raft_consensus.cc:504] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } } peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } }
I20260808 02:44:38.786190 5241 leader_election.cc:288] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 645595ba49ff4a60ac33aca9c703bc0a (127.25.137.193:46793), cee9bde7a81745e09b5449e36a323e39 (127.25.137.195:33541)
I20260808 02:44:38.793277 5229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.786660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:40665 (local address 127.25.137.193:46793)
0808 02:44:38.787025 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.787031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.787758 (+ 727us) server_negotiation.cc:407] Connection header received
0808 02:44:38.788020 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.788025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.788083 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.788172 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:38.788611 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.789371 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.791310 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.792369 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.792479 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.792531 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.792903 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.792987 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260808 02:44:38.793438 5246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.786538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46793 (local address 127.25.137.194:40665)
0808 02:44:38.787654 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:38.787670 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:38.787885 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:38.788203 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:38.788207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:38.788231 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:38.788495 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.788504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.789511 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.789516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.791135 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.791148 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.792570 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.792576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.792748 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.792784 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:44:38.792890 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 02:44:38.793036 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"spinlock_wait_cycles":16128,"thread_start_us":654,"threads_started":1}
I20260808 02:44:38.793875 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.786863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:46425 (local address 127.25.137.195:33541)
0808 02:44:38.787102 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.787109 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.787829 (+ 720us) server_negotiation.cc:407] Connection header received
0808 02:44:38.788018 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.788023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.788095 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.788175 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:38.789077 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.790069 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.791965 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.792929 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.793104 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.793174 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.793565 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.793692 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:44:38.794512 5247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.786683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:33541 (local address 127.25.137.194:46425)
0808 02:44:38.787699 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:38.787717 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:38.787880 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:38.788616 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:38.788619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:38.788638 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:38.788922 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.788930 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.790212 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.790217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.791778 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:38.791792 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.793180 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.793186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:38.793363 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.793400 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:44:38.794046 (+ 646us) client_negotiation.cc:240] Negotiation successful
0808 02:44:38.794169 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"spinlock_wait_cycles":2816,"thread_start_us":721,"threads_started":1}
I20260808 02:44:38.794468 4642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cb494a616e49a9af452e422275322c" candidate_uuid: "b2794284317c4a899881ef7bc6ce223f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "645595ba49ff4a60ac33aca9c703bc0a" is_pre_election: true
I20260808 02:44:38.795174 4642 raft_consensus.cc:2454] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2794284317c4a899881ef7bc6ce223f in term 0.
I20260808 02:44:38.795473 5099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cb494a616e49a9af452e422275322c" candidate_uuid: "b2794284317c4a899881ef7bc6ce223f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cee9bde7a81745e09b5449e36a323e39" is_pre_election: true
I20260808 02:44:38.795833 4772 leader_election.cc:302] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 645595ba49ff4a60ac33aca9c703bc0a, b2794284317c4a899881ef7bc6ce223f; no voters:
I20260808 02:44:38.795995 5099 raft_consensus.cc:2454] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2794284317c4a899881ef7bc6ce223f in term 0.
I20260808 02:44:38.796183 5241 raft_consensus.cc:2781] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:44:38.796311 5241 raft_consensus.cc:482] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:44:38.796417 5241 raft_consensus.cc:3037] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:38.798718 5241 raft_consensus.cc:504] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } } peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } }
I20260808 02:44:38.799296 5241 leader_election.cc:288] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [CANDIDATE]: Term 1 election: Requested vote from peers 645595ba49ff4a60ac33aca9c703bc0a (127.25.137.193:46793), cee9bde7a81745e09b5449e36a323e39 (127.25.137.195:33541)
I20260808 02:44:38.799691 4642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cb494a616e49a9af452e422275322c" candidate_uuid: "b2794284317c4a899881ef7bc6ce223f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "645595ba49ff4a60ac33aca9c703bc0a"
I20260808 02:44:38.799875 4642 raft_consensus.cc:3037] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:38.799836 5099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8cb494a616e49a9af452e422275322c" candidate_uuid: "b2794284317c4a899881ef7bc6ce223f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cee9bde7a81745e09b5449e36a323e39"
I20260808 02:44:38.800074 5099 raft_consensus.cc:3037] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:38.802372 4642 raft_consensus.cc:2454] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2794284317c4a899881ef7bc6ce223f in term 1.
I20260808 02:44:38.802556 5099 raft_consensus.cc:2454] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2794284317c4a899881ef7bc6ce223f in term 1.
I20260808 02:44:38.802827 4772 leader_election.cc:302] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 645595ba49ff4a60ac33aca9c703bc0a, b2794284317c4a899881ef7bc6ce223f; no voters:
I20260808 02:44:38.803169 5241 raft_consensus.cc:2781] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:38.803613 5241 raft_consensus.cc:686] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 1 LEADER]: Becoming Leader. State: Replica: b2794284317c4a899881ef7bc6ce223f, State: Running, Role: LEADER
I20260808 02:44:38.804136 5241 consensus_queue.cc:237] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } } peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } }
I20260808 02:44:38.809072 3133 catalog_manager.cc:5495] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f reported cstate change: term changed from 0 to 1, leader changed from <none> to b2794284317c4a899881ef7bc6ce223f (127.25.137.194). New cstate: current_term: 1 leader_uuid: "b2794284317c4a899881ef7bc6ce223f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } health_report { overall_health: UNKNOWN } } }
I20260808 02:44:38.873258 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.867162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.25.137.194:44887)
0808 02:44:38.867404 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.867409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.867428 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:44:38.867656 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.867660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.867772 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.867926 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:38.868342 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.869265 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.871109 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.871566 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.871656 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.872190 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:38.872644 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:38.872746 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.872900 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.872967 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 02:44:38.875483 4950 tablet.cc:2369] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:44:38.881471 4642 raft_consensus.cc:1261] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a [term 1 FOLLOWER]: Refusing update from remote peer b2794284317c4a899881ef7bc6ce223f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:38.881570 5099 raft_consensus.cc:1261] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39 [term 1 FOLLOWER]: Refusing update from remote peer b2794284317c4a899881ef7bc6ce223f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:38.882563 5248 consensus_queue.cc:1035] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [LEADER]: Connected to new peer: Peer: permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:38.882956 5241 consensus_queue.cc:1035] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:38.902226 5256 mvcc.cc:205] Tried to move back new op lower bound from 7316099395088760832 to 7316099394785521664. Current Snapshot: MvccSnapshot[applied={T|T < 7316099395088760832}]
I20260808 02:44:38.917632 3132 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:38.917938 3132 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:38.925092 3132 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 62b3c63d1651463a974424f583236de8: Sending DeleteTablet for 3 replicas of tablet c8cb494a616e49a9af452e422275322c
I20260808 02:44:38.926139 4843 tablet_service.cc:1514] Processing DeleteTablet for tablet c8cb494a616e49a9af452e422275322c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:38 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:50836
I20260808 02:44:38.926209 4605 tablet_service.cc:1514] Processing DeleteTablet for tablet c8cb494a616e49a9af452e422275322c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:38 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:35160
I20260808 02:44:38.926278 5068 tablet_service.cc:1514] Processing DeleteTablet for tablet c8cb494a616e49a9af452e422275322c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:38 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:57146
I20260808 02:44:38.939756 5270 tablet_replica.cc:330] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: stopping tablet replica
I20260808 02:44:38.940305 5270 raft_consensus.cc:2229] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:44:38.941098 5270 raft_consensus.cc:2258] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:38.941502 5271 tablet_replica.cc:330] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: stopping tablet replica
I20260808 02:44:38.941910 5272 tablet_replica.cc:330] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: stopping tablet replica
I20260808 02:44:38.941920 5271 raft_consensus.cc:2229] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:38.942301 5272 raft_consensus.cc:2229] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:38.942426 5271 raft_consensus.cc:2258] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:38.943001 5272 raft_consensus.cc:2258] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:38.944773 5270 ts_tablet_manager.cc:1905] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:38.944787 5271 ts_tablet_manager.cc:1905] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:38.944859 5272 ts_tablet_manager.cc:1905] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:38.947760 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 3085
I20260808 02:44:38.949000 3182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:38.951749 5270 ts_tablet_manager.cc:1918] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:38.951749 5272 ts_tablet_manager.cc:1918] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:38.951926 5272 log.cc:1196] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/c8cb494a616e49a9af452e422275322c
I20260808 02:44:38.951926 5270 log.cc:1196] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/c8cb494a616e49a9af452e422275322c
I20260808 02:44:38.952287 5271 ts_tablet_manager.cc:1918] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:38.952473 5271 log.cc:1196] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/c8cb494a616e49a9af452e422275322c
I20260808 02:44:38.952486 5270 ts_tablet_manager.cc:1939] T c8cb494a616e49a9af452e422275322c P b2794284317c4a899881ef7bc6ce223f: Deleting consensus metadata
I20260808 02:44:38.952489 5272 ts_tablet_manager.cc:1939] T c8cb494a616e49a9af452e422275322c P 645595ba49ff4a60ac33aca9c703bc0a: Deleting consensus metadata
I20260808 02:44:38.952993 5271 ts_tablet_manager.cc:1939] T c8cb494a616e49a9af452e422275322c P cee9bde7a81745e09b5449e36a323e39: Deleting consensus metadata
I20260808 02:44:38.971839 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.932031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57988 (local address 127.25.137.253:45953)
0808 02:44:38.932222 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.932228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.946368 (+ 14140us) server_negotiation.cc:407] Connection header received
0808 02:44:38.954520 (+ 8152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.954526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.954611 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.954700 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:38.957473 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.958593 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.960505 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.961187 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.961354 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.964993 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:38.965000 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:38.965032 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:38.965077 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:38.968614 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.969446 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.969452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.969456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.969612 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.970034 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.970039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.970043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.970400 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:38.970639 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.971222 (+ 583us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.971460 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:44:38.989494 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:38.947007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40766 (local address 127.25.137.252:42319)
0808 02:44:38.947194 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:38.947200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:38.972807 (+ 25607us) server_negotiation.cc:407] Connection header received
0808 02:44:38.973895 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:38.973901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:38.973985 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:38.974049 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:38.975860 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.976956 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.978781 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:38.979449 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:38.979564 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:38.982832 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:38.982839 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:38.982845 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:38.982886 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:38.986477 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.987247 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.987252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.987256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.987457 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:38.987925 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:38.987930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:38.987934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:38.988307 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:38.988575 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:38.988880 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 02:44:38.989131 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:44:39.081657 3109 catalog_manager.cc:4841] TS b2794284317c4a899881ef7bc6ce223f (127.25.137.194:44887): tablet c8cb494a616e49a9af452e422275322c (table test-table [id=dbff5a0ac9e54d68994099ae07156947]) successfully deleted
W20260808 02:44:39.082063 3109 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.25.137.194:44887, user_credentials={real_user=slave}} blocked reactor thread for 127836us
I20260808 02:44:39.082634 3112 catalog_manager.cc:4841] TS cee9bde7a81745e09b5449e36a323e39 (127.25.137.195:33541): tablet c8cb494a616e49a9af452e422275322c (table test-table [id=dbff5a0ac9e54d68994099ae07156947]) successfully deleted
W20260808 02:44:39.083091 3112 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.25.137.195:33541, user_credentials={real_user=slave}} blocked reactor thread for 128207us
I20260808 02:44:39.083606 3110 catalog_manager.cc:4841] TS 645595ba49ff4a60ac33aca9c703bc0a (127.25.137.193:46793): tablet c8cb494a616e49a9af452e422275322c (table test-table [id=dbff5a0ac9e54d68994099ae07156947]) successfully deleted
I20260808 02:44:39.084555 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 3085
W20260808 02:44:39.084548 3110 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.25.137.193:46793, user_credentials={real_user=slave}} blocked reactor thread for 129594us
I20260808 02:44:39.130492 5188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:39.114847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58002 (local address 127.25.137.253:45953)
0808 02:44:39.115059 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:39.115066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:39.115240 (+ 174us) server_negotiation.cc:407] Connection header received
0808 02:44:39.115542 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:39.115548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:39.115637 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:39.115738 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:39.117825 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:39.118643 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:39.120144 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:39.120835 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:39.120971 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:39.123767 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:39.123772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:39.123775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:39.123822 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:39.127433 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:39.128188 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:39.128193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:39.128196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:39.128350 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:39.128767 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:39.128772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:39.128776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:39.129156 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:39.129370 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:39.129944 (+ 574us) server_negotiation.cc:299] Negotiation successful
0808 02:44:39.130153 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:44:39.131119 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:39.115728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40782 (local address 127.25.137.252:42319)
0808 02:44:39.115954 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:39.115960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:39.116106 (+ 146us) server_negotiation.cc:407] Connection header received
0808 02:44:39.116268 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:39.116273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:39.116344 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:39.116423 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:39.118099 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:39.119226 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:39.120703 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:39.121303 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:39.121438 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:39.124815 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:39.124822 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:39.124827 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:39.124866 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:39.128106 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:39.128914 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:39.128920 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:39.128923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:39.129101 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:39.129496 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:39.129502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:39.129505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:39.129859 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:39.130052 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:39.130580 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 02:44:39.130790 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:44:39.904426 5286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:39.903465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.195:35671)
0808 02:44:39.904005 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:39.904163 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":265,"threads_started":1}
W20260808 02:44:39.904997 5162 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:34061 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
I20260808 02:44:39.906912 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:39.906121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.194:44463)
0808 02:44:39.906553 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:39.906699 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":226,"threads_started":1}
W20260808 02:44:39.907260 4944 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:34061 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
I20260808 02:44:39.911226 5288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:39.910518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.193:42237)
0808 02:44:39.910936 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:39.911039 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":205,"threads_started":1}
W20260808 02:44:39.911561 4711 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:34061 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
I20260808 02:44:40.425140 5289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 62b3c63d1651463a974424f583236de8)
I20260808 02:44:40.425447 5289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } }
I20260808 02:44:40.425839 5290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 62b3c63d1651463a974424f583236de8)
I20260808 02:44:40.426065 5290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } }
I20260808 02:44:40.426870 5289 leader_election.cc:288] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62b3c63d1651463a974424f583236de8 (127.25.137.254:34061), 90edc8737f924dfe82035aebcbc31b12 (127.25.137.253:45953)
I20260808 02:44:40.427347 5290 leader_election.cc:288] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62b3c63d1651463a974424f583236de8 (127.25.137.254:34061), 56fdbdbfc708421c8fed116fef9abb70 (127.25.137.252:42319)
I20260808 02:44:40.428153 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.427034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:33096)
0808 02:44:40.427701 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.427871 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":279,"threads_started":1}
I20260808 02:44:40.428363 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.427456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:33106)
0808 02:44:40.427926 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.428111 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":221,"threads_started":1}
W20260808 02:44:40.429308 4406 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
W20260808 02:44:40.429544 4088 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
I20260808 02:44:40.431725 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.431074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:33110)
0808 02:44:40.431313 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.431517 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260808 02:44:40.431864 4406 leader_election.cc:334] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 62b3c63d1651463a974424f583236de8 (127.25.137.254:34061): Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
I20260808 02:44:40.432700 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.432114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:33126)
0808 02:44:40.432347 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.432511 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 02:44:40.432850 4088 leader_election.cc:334] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 62b3c63d1651463a974424f583236de8 (127.25.137.254:34061): Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
I20260808 02:44:40.435899 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.427673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:45953 (local address 127.0.0.1:58012)
0808 02:44:40.428538 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.428555 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:40.428685 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:40.429650 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:40.429656 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:40.429897 (+ 241us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:40.430225 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:40.430241 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.432037 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.432042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:40.433808 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:40.433822 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.435325 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.435329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:40.435448 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.435473 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:44:40.435554 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 02:44:40.435656 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":1275,"threads_started":1}
I20260808 02:44:40.436024 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.428277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58012 (local address 127.25.137.253:45953)
0808 02:44:40.429261 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:40.429268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:40.429297 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:44:40.429370 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:40.429375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:40.429449 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:40.429609 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:40.430448 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.431861 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.433940 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.435158 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.435320 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.435391 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:40.435596 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 02:44:40.435727 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":595,"threads_started":1}
I20260808 02:44:40.436580 4128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56fdbdbfc708421c8fed116fef9abb70" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "90edc8737f924dfe82035aebcbc31b12" is_pre_election: true
I20260808 02:44:40.436872 4128 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56fdbdbfc708421c8fed116fef9abb70 in term 1.
I20260808 02:44:40.437405 4406 leader_election.cc:302] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56fdbdbfc708421c8fed116fef9abb70, 90edc8737f924dfe82035aebcbc31b12; no voters: 62b3c63d1651463a974424f583236de8
I20260808 02:44:40.437714 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.428052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:42319 (local address 127.0.0.1:40790)
0808 02:44:40.428566 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.428589 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:40.428726 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:40.432026 (+ 3300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:40.432032 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:40.432068 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:40.432375 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:40.432386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.433768 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.433772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:40.435662 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:40.435675 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.436951 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.436957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:40.437152 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.437187 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:44:40.437270 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 02:44:40.437410 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":1,"thread_start_us":1184,"threads_started":1}
I20260808 02:44:40.437995 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.430120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40790 (local address 127.25.137.252:42319)
0808 02:44:40.431100 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:40.431106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:40.431131 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:44:40.431195 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:40.431200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:40.431269 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:40.432493 (+ 1224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:40.432706 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.433629 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.435832 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.436752 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.436913 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.436963 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:40.437324 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 02:44:40.437459 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":710,"threads_started":1}
I20260808 02:44:40.437973 5289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 02:44:40.438148 5289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 62b3c63d1651463a974424f583236de8)
I20260808 02:44:40.438253 5289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:44:40.438540 4442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90edc8737f924dfe82035aebcbc31b12" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "56fdbdbfc708421c8fed116fef9abb70" is_pre_election: true
I20260808 02:44:40.440481 5289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } }
I20260808 02:44:40.440949 4442 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90edc8737f924dfe82035aebcbc31b12 in current term 2: Already voted for candidate 56fdbdbfc708421c8fed116fef9abb70 in this term.
I20260808 02:44:40.441113 5289 leader_election.cc:288] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [CANDIDATE]: Term 2 election: Requested vote from peers 62b3c63d1651463a974424f583236de8 (127.25.137.254:34061), 90edc8737f924dfe82035aebcbc31b12 (127.25.137.253:45953)
I20260808 02:44:40.441553 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.441239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:33130)
0808 02:44:40.441354 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.441419 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 02:44:40.441824 4128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56fdbdbfc708421c8fed116fef9abb70" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "90edc8737f924dfe82035aebcbc31b12"
I20260808 02:44:40.442042 4128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:44:40.442277 4088 leader_election.cc:302] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90edc8737f924dfe82035aebcbc31b12; no voters: 56fdbdbfc708421c8fed116fef9abb70, 62b3c63d1651463a974424f583236de8
I20260808 02:44:40.442600 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.442197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:33142)
0808 02:44:40.442361 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.442465 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 02:44:40.442798 4406 leader_election.cc:334] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 62b3c63d1651463a974424f583236de8 (127.25.137.254:34061): Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
I20260808 02:44:40.444259 4128 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56fdbdbfc708421c8fed116fef9abb70 in term 2.
I20260808 02:44:40.444453 5290 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:44:40.444638 4406 leader_election.cc:302] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56fdbdbfc708421c8fed116fef9abb70, 90edc8737f924dfe82035aebcbc31b12; no voters: 62b3c63d1651463a974424f583236de8
I20260808 02:44:40.444886 5289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:44:40.445361 5289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [term 2 LEADER]: Becoming Leader. State: Replica: 56fdbdbfc708421c8fed116fef9abb70, State: Running, Role: LEADER
I20260808 02:44:40.445819 5289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } }
I20260808 02:44:40.447371 5304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56fdbdbfc708421c8fed116fef9abb70. Latest consensus state: current_term: 2 leader_uuid: "56fdbdbfc708421c8fed116fef9abb70" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:40.447678 5304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:40.448325 5307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:40.452445 5307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:40.453334 5307 catalog_manager.cc:1259] Loaded cluster ID: 8e115c5b06734bc486ed5cec4912f3fa
I20260808 02:44:40.453415 5307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:40.454558 5307 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:40.455381 5307 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: Loaded TSK: 0
I20260808 02:44:40.456280 5307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:44:40.477351 4429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:44:40.488085 4429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:44:40.493116 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.492623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:33158)
0808 02:44:40.492842 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.492918 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260808 02:44:40.493203 4128 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [term 2 FOLLOWER]: Refusing update from remote peer 56fdbdbfc708421c8fed116fef9abb70: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 02:44:40.494254 5304 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 02:44:40.494489 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.493775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:33172)
0808 02:44:40.494224 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.494313 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350}
W20260808 02:44:40.494658 4406 consensus_peers.cc:487] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 -> Peer 62b3c63d1651463a974424f583236de8 (127.25.137.254:34061): Couldn't send request to peer 62b3c63d1651463a974424f583236de8. Status: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 02:44:40.499260 5290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56fdbdbfc708421c8fed116fef9abb70. Latest consensus state: current_term: 2 leader_uuid: "56fdbdbfc708421c8fed116fef9abb70" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:40.499641 5290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:44:40.501569 5289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "56fdbdbfc708421c8fed116fef9abb70" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:40.501812 5289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:40.502733 5290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "56fdbdbfc708421c8fed116fef9abb70" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "62b3c63d1651463a974424f583236de8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34061 } } peers { permanent_uuid: "90edc8737f924dfe82035aebcbc31b12" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "56fdbdbfc708421c8fed116fef9abb70" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 42319 } attrs { promote: false } } }
I20260808 02:44:40.503032 5290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90edc8737f924dfe82035aebcbc31b12 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:44:40.519114 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.511818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:46793 (local address 127.0.0.1:35174)
0808 02:44:40.512016 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.512035 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:40.512141 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:40.512934 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:40.512940 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:40.512967 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:40.513204 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:40.513218 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.514834 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.514840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:40.516640 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:40.516655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.518289 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.518295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:40.518475 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.518511 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:44:40.518609 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 02:44:40.518764 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 02:44:40.519748 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.512105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35174 (local address 127.25.137.193:46793)
0808 02:44:40.512590 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:40.512597 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:40.512616 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:44:40.512692 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:40.512697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:40.512771 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:40.513373 (+ 602us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:40.513549 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.514688 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.516869 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.518124 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.518314 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.518400 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:40.519328 (+ 928us) server_negotiation.cc:299] Negotiation successful
0808 02:44:40.519479 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":1046,"threads_started":1}
I20260808 02:44:40.520617 4605 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb07f5d19d641beb1e38f1eb7f48044 (DEFAULT_TABLE table=test-table [id=b3ac0c298bd345cd967f7c1772f65505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:40.521173 4605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb07f5d19d641beb1e38f1eb7f48044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:40.522814 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.512586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:33541 (local address 127.0.0.1:57160)
0808 02:44:40.512818 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.512837 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:40.512950 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:40.513905 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:40.513910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:40.513934 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:40.514192 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:40.514201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.516687 (+ 2486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.516691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:40.518629 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:40.518646 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.522072 (+ 3426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.522078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:40.522251 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.522275 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:44:40.522366 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 02:44:40.522507 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260808 02:44:40.524528 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.512628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:44887 (local address 127.0.0.1:50860)
0808 02:44:40.517944 (+ 5316us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.517963 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:40.518085 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:40.518550 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:40.518554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:40.518583 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:40.518966 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:40.518979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.520353 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.520358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:40.522251 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:40.522267 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.523725 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.523731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:40.523915 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.523948 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:44:40.524040 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 02:44:40.524178 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5173,"thread_start_us":4936,"threads_started":1}
I20260808 02:44:40.524777 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.512647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57160 (local address 127.25.137.195:33541)
0808 02:44:40.513588 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:40.513594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:40.513612 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:44:40.513683 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:40.513689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:40.513768 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:40.513869 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:40.514340 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.516528 (+ 2188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.520912 (+ 4384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.521862 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.522023 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.522099 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:40.523798 (+ 1699us) server_negotiation.cc:299] Negotiation successful
0808 02:44:40.523973 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":729,"threads_started":1}
I20260808 02:44:40.525480 5328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.513057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50860 (local address 127.25.137.194:44887)
0808 02:44:40.517402 (+ 4345us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:40.517408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:40.518058 (+ 650us) server_negotiation.cc:407] Connection header received
0808 02:44:40.518199 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:40.518205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:40.518284 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:40.518375 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:40.519178 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.520173 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.522418 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.523520 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.523672 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.523725 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:40.524975 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0808 02:44:40.525177 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4197,"thread_start_us":4098,"threads_started":1}
I20260808 02:44:40.525462 5068 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb07f5d19d641beb1e38f1eb7f48044 (DEFAULT_TABLE table=test-table [id=b3ac0c298bd345cd967f7c1772f65505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:40.525918 5068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb07f5d19d641beb1e38f1eb7f48044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:40.526378 4843 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb07f5d19d641beb1e38f1eb7f48044 (DEFAULT_TABLE table=test-table [id=b3ac0c298bd345cd967f7c1772f65505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:40.526813 4843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb07f5d19d641beb1e38f1eb7f48044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:40.531798 5330 tablet_bootstrap.cc:492] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a: Bootstrap starting.
I20260808 02:44:40.533775 5330 tablet_bootstrap.cc:654] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:40.537590 5332 tablet_bootstrap.cc:492] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f: Bootstrap starting.
I20260808 02:44:40.539057 5330 tablet_bootstrap.cc:492] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a: No bootstrap required, opened a new log
I20260808 02:44:40.539218 5330 ts_tablet_manager.cc:1397] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 02:44:40.539980 5330 raft_consensus.cc:348] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } } peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } }
I20260808 02:44:40.540194 5330 raft_consensus.cc:374] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:40.540287 5330 raft_consensus.cc:729] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 645595ba49ff4a60ac33aca9c703bc0a, State: Initialized, Role: FOLLOWER
I20260808 02:44:40.540560 5330 consensus_queue.cc:260] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } } peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } }
I20260808 02:44:40.541348 5332 tablet_bootstrap.cc:654] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:40.542991 5331 tablet_bootstrap.cc:492] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39: Bootstrap starting.
I20260808 02:44:40.544400 5331 tablet_bootstrap.cc:654] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:40.545323 5330 ts_tablet_manager.cc:1428] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260808 02:44:40.546398 5332 tablet_bootstrap.cc:492] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f: No bootstrap required, opened a new log
I20260808 02:44:40.546614 5332 ts_tablet_manager.cc:1397] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260808 02:44:40.546666 5331 tablet_bootstrap.cc:492] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39: No bootstrap required, opened a new log
I20260808 02:44:40.546808 5331 ts_tablet_manager.cc:1397] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 02:44:40.546888 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.541603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.193:57761)
0808 02:44:40.545889 (+ 4286us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.546609 (+ 720us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4158,"thread_start_us":4030,"threads_started":1}
I20260808 02:44:40.547605 5331 raft_consensus.cc:348] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } } peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } }
I20260808 02:44:40.547614 5332 raft_consensus.cc:348] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } } peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } }
I20260808 02:44:40.547847 5331 raft_consensus.cc:374] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:40.547935 5331 raft_consensus.cc:729] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cee9bde7a81745e09b5449e36a323e39, State: Initialized, Role: FOLLOWER
I20260808 02:44:40.548110 5332 raft_consensus.cc:374] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:40.548250 5332 raft_consensus.cc:729] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2794284317c4a899881ef7bc6ce223f, State: Initialized, Role: FOLLOWER
I20260808 02:44:40.548180 5331 consensus_queue.cc:260] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } } peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } }
I20260808 02:44:40.548614 5332 consensus_queue.cc:260] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } } peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } }
I20260808 02:44:40.549300 4712 heartbeater.cc:503] Master 127.25.137.252:42319 was elected leader, sending a full tablet report...
I20260808 02:44:40.549556 5331 ts_tablet_manager.cc:1428] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 02:44:40.551050 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.548891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.193:39491)
0808 02:44:40.550792 (+ 1901us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.550898 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1784}
I20260808 02:44:40.555464 5164 heartbeater.cc:503] Master 127.25.137.252:42319 was elected leader, sending a full tablet report...
I20260808 02:44:40.556087 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.549815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.195:38457)
0808 02:44:40.553943 (+ 4128us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.555812 (+ 1869us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4000,"thread_start_us":3879,"threads_started":1}
I20260808 02:44:40.558135 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.557531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.195:41723)
0808 02:44:40.557907 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.558000 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271}
I20260808 02:44:40.558614 5341 raft_consensus.cc:482] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:40.559055 5332 ts_tablet_manager.cc:1428] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f: Time spent starting tablet: real 0.012s user 0.006s sys 0.004s
I20260808 02:44:40.558879 5341 raft_consensus.cc:504] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } } peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } }
I20260808 02:44:40.559473 5344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.551419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.194:35643)
0808 02:44:40.559119 (+ 7700us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.559260 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7572,"threads_started":1}
I20260808 02:44:40.559624 5341 leader_election.cc:288] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 645595ba49ff4a60ac33aca9c703bc0a (127.25.137.193:46793), cee9bde7a81745e09b5449e36a323e39 (127.25.137.195:33541)
I20260808 02:44:40.560108 4642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb07f5d19d641beb1e38f1eb7f48044" candidate_uuid: "b2794284317c4a899881ef7bc6ce223f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "645595ba49ff4a60ac33aca9c703bc0a" is_pre_election: true
I20260808 02:44:40.560179 5099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb07f5d19d641beb1e38f1eb7f48044" candidate_uuid: "b2794284317c4a899881ef7bc6ce223f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cee9bde7a81745e09b5449e36a323e39" is_pre_election: true
I20260808 02:44:40.560369 4642 raft_consensus.cc:2454] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2794284317c4a899881ef7bc6ce223f in term 0.
I20260808 02:44:40.560451 5099 raft_consensus.cc:2454] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2794284317c4a899881ef7bc6ce223f in term 0.
I20260808 02:44:40.560885 4772 leader_election.cc:302] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 645595ba49ff4a60ac33aca9c703bc0a, b2794284317c4a899881ef7bc6ce223f; no voters:
I20260808 02:44:40.561342 4946 heartbeater.cc:503] Master 127.25.137.252:42319 was elected leader, sending a full tablet report...
I20260808 02:44:40.562132 5341 raft_consensus.cc:2781] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:44:40.562356 5344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.561848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.194:36017)
0808 02:44:40.561987 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.562084 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 02:44:40.562376 5341 raft_consensus.cc:482] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:44:40.562633 5341 raft_consensus.cc:3037] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:40.564719 5341 raft_consensus.cc:504] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } } peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } }
I20260808 02:44:40.565367 5341 leader_election.cc:288] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [CANDIDATE]: Term 1 election: Requested vote from peers 645595ba49ff4a60ac33aca9c703bc0a (127.25.137.193:46793), cee9bde7a81745e09b5449e36a323e39 (127.25.137.195:33541)
I20260808 02:44:40.565800 4642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb07f5d19d641beb1e38f1eb7f48044" candidate_uuid: "b2794284317c4a899881ef7bc6ce223f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "645595ba49ff4a60ac33aca9c703bc0a"
I20260808 02:44:40.565843 5099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb07f5d19d641beb1e38f1eb7f48044" candidate_uuid: "b2794284317c4a899881ef7bc6ce223f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cee9bde7a81745e09b5449e36a323e39"
I20260808 02:44:40.566031 4642 raft_consensus.cc:3037] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:40.566030 5099 raft_consensus.cc:3037] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:40.568866 5099 raft_consensus.cc:2454] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2794284317c4a899881ef7bc6ce223f in term 1.
I20260808 02:44:40.568866 4642 raft_consensus.cc:2454] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2794284317c4a899881ef7bc6ce223f in term 1.
I20260808 02:44:40.569258 4772 leader_election.cc:302] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 645595ba49ff4a60ac33aca9c703bc0a, b2794284317c4a899881ef7bc6ce223f; no voters:
I20260808 02:44:40.569511 5341 raft_consensus.cc:2781] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:40.569658 5341 raft_consensus.cc:686] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 1 LEADER]: Becoming Leader. State: Replica: b2794284317c4a899881ef7bc6ce223f, State: Running, Role: LEADER
I20260808 02:44:40.569937 5341 consensus_queue.cc:237] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } } peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } }
I20260808 02:44:40.570516 5344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.570079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.194:39095)
0808 02:44:40.570235 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.570326 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 02:44:40.572781 4429 catalog_manager.cc:5495] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f reported cstate change: term changed from 0 to 1, leader changed from <none> to b2794284317c4a899881ef7bc6ce223f (127.25.137.194). New cstate: current_term: 1 leader_uuid: "b2794284317c4a899881ef7bc6ce223f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2794284317c4a899881ef7bc6ce223f" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 44887 } health_report { overall_health: HEALTHY } } }
I20260808 02:44:40.590804 5328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.585886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50862 (local address 127.25.137.194:44887)
0808 02:44:40.586123 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:40.586128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:40.586144 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:44:40.586268 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:40.586272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:40.586340 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:40.586410 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:40.586903 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.587691 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.589117 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.589545 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.589652 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.590050 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:40.590169 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:40.590261 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:40.590415 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 02:44:40.590489 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260808 02:44:40.594060 5099 raft_consensus.cc:1261] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39 [term 1 FOLLOWER]: Refusing update from remote peer b2794284317c4a899881ef7bc6ce223f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:40.594095 4642 raft_consensus.cc:1261] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a [term 1 FOLLOWER]: Refusing update from remote peer b2794284317c4a899881ef7bc6ce223f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:40.594796 5340 consensus_queue.cc:1035] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [LEADER]: Connected to new peer: Peer: permanent_uuid: "645595ba49ff4a60ac33aca9c703bc0a" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:40.595125 5341 consensus_queue.cc:1035] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cee9bde7a81745e09b5449e36a323e39" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:40.597673 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.596960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.193:51167)
0808 02:44:40.597418 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.597539 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328}
I20260808 02:44:40.598717 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.598074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.195:45323)
0808 02:44:40.598335 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.598476 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260808 02:44:40.606557 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.605994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.193:57365)
0808 02:44:40.606254 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.606358 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260808 02:44:40.612771 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.612139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.195:35181)
0808 02:44:40.612429 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.612534 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260808 02:44:40.623502 5344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.623058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.25.137.194:47581)
0808 02:44:40.623238 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.623339 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 02:44:40.627879 4430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:40.628156 4430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:40.633352 4430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56fdbdbfc708421c8fed116fef9abb70: Sending DeleteTablet for 3 replicas of tablet 9cb07f5d19d641beb1e38f1eb7f48044
I20260808 02:44:40.634347 4605 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb07f5d19d641beb1e38f1eb7f48044 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:40 UTC) from {username='oryx', principal='oryx/127.25.137.252@KRBTEST.COM'} at 127.0.0.1:35174
I20260808 02:44:40.634397 4843 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb07f5d19d641beb1e38f1eb7f48044 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:40 UTC) from {username='oryx', principal='oryx/127.25.137.252@KRBTEST.COM'} at 127.0.0.1:50860
I20260808 02:44:40.634492 5068 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb07f5d19d641beb1e38f1eb7f48044 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:40 UTC) from {username='oryx', principal='oryx/127.25.137.252@KRBTEST.COM'} at 127.0.0.1:57160
I20260808 02:44:40.635927 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 4394
I20260808 02:44:40.641450 5365 tablet_replica.cc:330] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a: stopping tablet replica
I20260808 02:44:40.641642 5365 raft_consensus.cc:2229] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:40.641924 5365 raft_consensus.cc:2258] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:40.642526 5367 tablet_replica.cc:330] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f: stopping tablet replica
I20260808 02:44:40.642712 5367 raft_consensus.cc:2229] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:44:40.643091 5365 ts_tablet_manager.cc:1905] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:40.643185 5367 raft_consensus.cc:2258] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:40.643761 5368 tablet_replica.cc:330] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39: stopping tablet replica
I20260808 02:44:40.643965 5368 raft_consensus.cc:2229] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:40.644243 5368 raft_consensus.cc:2258] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:40.645212 5367 ts_tablet_manager.cc:1905] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:40.649753 5365 ts_tablet_manager.cc:1918] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:40.649923 5365 log.cc:1196] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/9cb07f5d19d641beb1e38f1eb7f48044
I20260808 02:44:40.649973 5368 ts_tablet_manager.cc:1905] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:40.650419 5365 ts_tablet_manager.cc:1939] T 9cb07f5d19d641beb1e38f1eb7f48044 P 645595ba49ff4a60ac33aca9c703bc0a: Deleting consensus metadata
I20260808 02:44:40.651697 5367 ts_tablet_manager.cc:1918] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:40.651678 4405 catalog_manager.cc:4841] TS 645595ba49ff4a60ac33aca9c703bc0a (127.25.137.193:46793): tablet 9cb07f5d19d641beb1e38f1eb7f48044 (table test-table [id=b3ac0c298bd345cd967f7c1772f65505]) successfully deleted
I20260808 02:44:40.651858 5367 log.cc:1196] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/9cb07f5d19d641beb1e38f1eb7f48044
I20260808 02:44:40.652464 5367 ts_tablet_manager.cc:1939] T 9cb07f5d19d641beb1e38f1eb7f48044 P b2794284317c4a899881ef7bc6ce223f: Deleting consensus metadata
I20260808 02:44:40.653544 4403 catalog_manager.cc:4841] TS b2794284317c4a899881ef7bc6ce223f (127.25.137.194:44887): tablet 9cb07f5d19d641beb1e38f1eb7f48044 (table test-table [id=b3ac0c298bd345cd967f7c1772f65505]) successfully deleted
I20260808 02:44:40.655727 5368 ts_tablet_manager.cc:1918] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:40.655856 5368 log.cc:1196] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/9cb07f5d19d641beb1e38f1eb7f48044
I20260808 02:44:40.656201 5368 ts_tablet_manager.cc:1939] T 9cb07f5d19d641beb1e38f1eb7f48044 P cee9bde7a81745e09b5449e36a323e39: Deleting consensus metadata
I20260808 02:44:40.657120 4408 catalog_manager.cc:4841] TS cee9bde7a81745e09b5449e36a323e39 (127.25.137.195:33541): tablet 9cb07f5d19d641beb1e38f1eb7f48044 (table test-table [id=b3ac0c298bd345cd967f7c1772f65505]) successfully deleted
I20260808 02:44:40.659492 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.637058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35176 (local address 127.25.137.193:46793)
0808 02:44:40.637368 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:40.637375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:40.645548 (+ 8173us) server_negotiation.cc:407] Connection header received
0808 02:44:40.645711 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:40.645717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:40.645784 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:40.645865 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:40.647194 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.647944 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.649270 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.649918 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.650024 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.653951 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:40.653967 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:40.653984 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:40.654016 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:40.657162 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:40.657791 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:40.657797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:40.657800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:40.657938 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:40.658228 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:40.658231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:40.658233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:40.658595 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:40.658818 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:40.659026 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 02:44:40.659195 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 02:44:40.660113 4704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:40.752246 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 4394
I20260808 02:44:40.778522 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 4728
I20260808 02:44:40.792280 5328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.779247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.25.137.194:44887)
0808 02:44:40.779441 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:40.779447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:40.779467 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:44:40.779656 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:40.779661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:40.779732 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:40.779815 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:40.781127 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.781880 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.783110 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.783512 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.783615 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.786244 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:40.786262 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:40.786279 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:40.786322 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:40.789472 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:40.790154 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:40.790163 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:40.790165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:40.790307 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:40.790635 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:40.790638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:40.790640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:40.791053 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:40.791267 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:40.791647 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 02:44:40.791978 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:44:40.792972 4939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:40.924821 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 4728
I20260808 02:44:40.936370 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.935793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:33180)
0808 02:44:40.936037 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.936165 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:44:40.937583 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.937246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34061 (local address 127.0.0.1:33182)
0808 02:44:40.937384 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:40.937462 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:34061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 02:44:40.952812 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 4967
I20260808 02:44:40.966245 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:40.953666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57166 (local address 127.25.137.195:33541)
0808 02:44:40.953899 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:40.953905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:40.953927 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:44:40.953997 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:40.954004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:40.954074 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:40.954169 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:40.955541 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.956267 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.957620 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:40.958041 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:40.958156 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:40.960697 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:40.960714 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:40.960732 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:40.960768 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:40.963517 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:40.964172 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:40.964180 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:40.964183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:40.964310 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:40.964678 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:40.964681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:40.964684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:40.965105 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:40.965400 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:40.965703 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 02:44:40.965945 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:44:40.967054 5155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:41.062096 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 4967
I20260808 02:44:41.090379 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 3890
I20260808 02:44:41.091528 4166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:41.211427 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 3890
I20260808 02:44:41.240926 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 4050
I20260808 02:44:41.242038 4473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:41.366155 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 4050
2026-08-08T02:44:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9641 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:44:41 dist-test-slave-k5rr krb5kdc[5379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:44:41 dist-test-slave-k5rr krb5kdc[5379](info): set up 2 sockets
Aug 08 02:44:41 dist-test-slave-k5rr krb5kdc[5379](info): commencing operation
krb5kdc: starting...
W20260808 02:44:42.606196 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.004s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:44:42 dist-test-slave-k5rr krb5kdc[5379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:44:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.254.keytab.
Entry for principal oryx/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.254.keytab.
I20260808 02:44:42.863191 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:36945
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.254.keytab
--principal=oryx/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:36945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:43.204206 5406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:43.204494 5406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:43.204627 5406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:43.211830 5406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:43.211941 5406 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:44:43.211992 5406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:43.212078 5406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:43.212137 5406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:43.212229 5406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:43.222589 5406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:36945
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.254.keytab
--principal=oryx/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:36945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:43.224000 5406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:43.225495 5406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:43.237382 5414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:43.237443 5411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:43.238348 5417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:43.238607 5406 server_base.cc:1034] running on GCE node
I20260808 02:44:43.239415 5406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:43.240813 5406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:43.242048 5406 hybrid_clock.cc:648] HybridClock initialized: now 1786157083241965 us; error 53 us; skew 500 ppm
I20260808 02:44:43.242388 5406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:43 dist-test-slave-k5rr krb5kdc[5379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157083, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:43.245474 5406 init.cc:377] Logged in from keytab as oryx/127.25.137.254@KRBTEST.COM (short username oryx)
I20260808 02:44:43.247013 5406 webserver.cc:460] Webserver started at http://127.25.137.254:42085/ using document root <none> and password file <none>
I20260808 02:44:43.247705 5406 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:43.247809 5406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:43.248133 5406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:43.251304 5406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "9ac89efa23664878b1546a04e24c67d5"
format_stamp: "Formatted at 2026-08-08 02:44:43 on dist-test-slave-k5rr"
server_key: "70469dd3c58c94bab9f9b608bcc5f525"
server_key_iv: "8a1e740119b328fe0633c5b1c870d704"
server_key_version: "encryptionkey@0"
I20260808 02:44:43.252223 5406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "9ac89efa23664878b1546a04e24c67d5"
format_stamp: "Formatted at 2026-08-08 02:44:43 on dist-test-slave-k5rr"
server_key: "70469dd3c58c94bab9f9b608bcc5f525"
server_key_iv: "8a1e740119b328fe0633c5b1c870d704"
server_key_version: "encryptionkey@0"
I20260808 02:44:43.257624 5406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:44:43.261195 5425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:43.262693 5406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:43.262864 5406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "9ac89efa23664878b1546a04e24c67d5"
format_stamp: "Formatted at 2026-08-08 02:44:43 on dist-test-slave-k5rr"
server_key: "70469dd3c58c94bab9f9b608bcc5f525"
server_key_iv: "8a1e740119b328fe0633c5b1c870d704"
server_key_version: "encryptionkey@0"
I20260808 02:44:43.263082 5406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:43.276137 5406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:43.280371 5406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:43.280620 5406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:43.301335 5406 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:36945
I20260808 02:44:43.301358 5531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:36945 every 8 connection(s)
I20260808 02:44:43.302649 5406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:43.305153 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 5406
I20260808 02:44:43.305368 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:43.306039 26151 external_mini_cluster.cc:1428] Setting key 5a6cb7f9efa6be9093d39c2296efdf0f
I20260808 02:44:43.306613 5532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 02:44:43 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:43.321292 5532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5: Bootstrap starting.
I20260808 02:44:43.325625 5532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:43.326849 5532 log.cc:824] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:43.330344 5538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:43.307878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49078 (local address 127.25.137.254:36945)
0808 02:44:43.308626 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:43.308642 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:43.308693 (+ 51us) server_negotiation.cc:407] Connection header received
0808 02:44:43.310538 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:43.310566 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:43.311086 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:43.311568 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:43.312835 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:43.314952 (+ 2117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:43.316187 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:43.317278 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:43.317476 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:43.321453 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:43.321465 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:43.321484 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:43.321532 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:43.325520 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:43.326221 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:43.326229 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:43.326237 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:43.326461 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:43.326905 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:43.326911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:43.326914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:43.327679 (+ 765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:43.327922 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:43.328465 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 02:44:43.328979 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":280,"threads_started":1}
I20260808 02:44:43.330379 5532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5: No bootstrap required, opened a new log
I20260808 02:44:43.338145 5532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac89efa23664878b1546a04e24c67d5" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36945 } }
I20260808 02:44:43.338500 5532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:43.338581 5532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac89efa23664878b1546a04e24c67d5, State: Initialized, Role: FOLLOWER
I20260808 02:44:43.339084 5532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac89efa23664878b1546a04e24c67d5" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36945 } }
I20260808 02:44:43.339304 5532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:43.339422 5532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:43.339586 5532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:43.341881 5532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac89efa23664878b1546a04e24c67d5" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36945 } }
I20260808 02:44:43.342391 5532 leader_election.cc:302] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ac89efa23664878b1546a04e24c67d5; no voters:
I20260808 02:44:43.342939 5532 leader_election.cc:288] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:43.343065 5542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:43.343703 5542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [term 1 LEADER]: Becoming Leader. State: Replica: 9ac89efa23664878b1546a04e24c67d5, State: Running, Role: LEADER
I20260808 02:44:43.344206 5542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac89efa23664878b1546a04e24c67d5" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36945 } }
I20260808 02:44:43.345067 5532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:43.346712 5543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ac89efa23664878b1546a04e24c67d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac89efa23664878b1546a04e24c67d5" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36945 } } }
I20260808 02:44:43.347096 5543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:43.347528 5544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ac89efa23664878b1546a04e24c67d5. Latest consensus state: current_term: 1 leader_uuid: "9ac89efa23664878b1546a04e24c67d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac89efa23664878b1546a04e24c67d5" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36945 } } }
I20260808 02:44:43.347822 5544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:43.349532 5548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:43.354522 5548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:43.362398 5548 catalog_manager.cc:1347] Generated new cluster ID: 2f577ffa2dcf40c791e6c3827622a3d5
I20260808 02:44:43.362505 5548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:43.389570 5548 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:43.391609 5548 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:43.402271 5548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5: Generated new TSK 0
I20260808 02:44:43.403383 5548 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab.
Entry for principal oryx/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab.
I20260808 02:44:43.466658 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:36945
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab
--principal=oryx/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:43.810791 5571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:43.811069 5571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:43.811223 5571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:43.818552 5571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:43.818670 5571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:43.818806 5571 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:44:43.818933 5571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:44:43.828634 5571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab
--principal=oryx/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.137.254:36945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:43.830036 5571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:43.831377 5571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:43.844554 5581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:43.844717 5580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:43.844722 5583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:43.845760 5571 server_base.cc:1034] running on GCE node
I20260808 02:44:43.846479 5571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:43.847440 5571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:43.848671 5571 hybrid_clock.cc:648] HybridClock initialized: now 1786157083848595 us; error 52 us; skew 500 ppm
I20260808 02:44:43.848994 5571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:43 dist-test-slave-k5rr krb5kdc[5379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157083, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:43.852234 5571 init.cc:377] Logged in from keytab as oryx/127.25.137.193@KRBTEST.COM (short username oryx)
I20260808 02:44:43.853704 5571 webserver.cc:460] Webserver started at http://127.25.137.193:35467/ using document root <none> and password file <none>
I20260808 02:44:43.854441 5571 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:43.854560 5571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:43.854902 5571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:43.857951 5571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "8afcd23c51b249c3912614f4d62f07d4"
format_stamp: "Formatted at 2026-08-08 02:44:43 on dist-test-slave-k5rr"
server_key: "e29cce7b419b8e6aa16525633b15bc24"
server_key_iv: "784dc5edb76bc0b21e7e5cf33a85f819"
server_key_version: "encryptionkey@0"
I20260808 02:44:43.858793 5571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "8afcd23c51b249c3912614f4d62f07d4"
format_stamp: "Formatted at 2026-08-08 02:44:43 on dist-test-slave-k5rr"
server_key: "e29cce7b419b8e6aa16525633b15bc24"
server_key_iv: "784dc5edb76bc0b21e7e5cf33a85f819"
server_key_version: "encryptionkey@0"
I20260808 02:44:43.864027 5571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 02:44:43.867589 5592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:43.868927 5571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:44:43.869141 5571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "8afcd23c51b249c3912614f4d62f07d4"
format_stamp: "Formatted at 2026-08-08 02:44:43 on dist-test-slave-k5rr"
server_key: "e29cce7b419b8e6aa16525633b15bc24"
server_key_iv: "784dc5edb76bc0b21e7e5cf33a85f819"
server_key_version: "encryptionkey@0"
I20260808 02:44:43.869369 5571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:43.882833 5571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:43.887079 5571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:43.887369 5571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:43.890285 5571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:43.890379 5571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:43.890586 5571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:43.890661 5571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:44:43 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157083, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:43.912494 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:43.890708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:48363 (local address 127.25.137.254:36945)
0808 02:44:43.891289 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:43.891296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:43.892512 (+ 1216us) server_negotiation.cc:407] Connection header received
0808 02:44:43.895416 (+ 2904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:43.895423 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:43.895503 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:43.895598 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:43.898096 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:43.898971 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:43.901397 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:43.901979 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:43.902095 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:43.906425 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:43.906430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:43.906433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:43.906466 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:43.908812 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:43.909711 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:43.909715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:43.909718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:43.909855 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:43.910477 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:43.910482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:43.910485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:43.910825 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:43.910981 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:43.912035 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0808 02:44:43.912231 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":276,"threads_started":1}
I20260808 02:44:43.914896 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:43.891336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:36945 (local address 127.25.137.193:48363)
0808 02:44:43.892305 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:43.892370 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:43.895052 (+ 2682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:43.895909 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:43.895926 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:43.896777 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:43.897860 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:43.897893 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:43.899130 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:43.899135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:43.901197 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:43.901217 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:43.902233 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:43.902242 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:43.902675 (+ 433us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:43.903428 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:43.903463 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:43.906193 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:43.908948 (+ 2755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:43.908957 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:43.908973 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:43.909581 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:43.909966 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:43.909972 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:43.909976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:43.910357 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:43.911047 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:43.911053 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:43.911556 (+ 503us) client_negotiation.cc:769] Sending connection context
0808 02:44:43.912770 (+ 1214us) client_negotiation.cc:240] Negotiation successful
0808 02:44:43.913360 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":331,"threads_started":1}
I20260808 02:44:43.936115 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:43.921977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:36945 (local address 127.25.137.193:55927)
0808 02:44:43.922673 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:43.922698 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:43.922862 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:43.923174 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:43.923179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:43.923848 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:43.924297 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:43.924313 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:43.925353 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:43.925359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:43.926903 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:43.926926 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:43.927792 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:43.927799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:43.927955 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:43.928619 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:43.928639 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:43.929798 (+ 1159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:43.933036 (+ 3238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:43.933042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:43.933046 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:43.933659 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:43.934076 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:43.934081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:43.934084 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:43.934253 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:43.934850 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:43.934855 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:43.935084 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 02:44:43.935276 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 02:44:43.935581 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":342,"threads_started":1}
I20260808 02:44:43.936309 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:43.922015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:55927 (local address 127.25.137.254:36945)
0808 02:44:43.922192 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:43.922199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:43.922788 (+ 589us) server_negotiation.cc:407] Connection header received
0808 02:44:43.922984 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:43.922988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:43.923048 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:43.923123 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:43.924452 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:43.925189 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:43.927045 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:43.927595 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:43.927703 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:43.929976 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:43.929982 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:43.929986 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:43.930017 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:43.932910 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:43.933764 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:43.933769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:43.933773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:43.933929 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:43.934365 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:43.934370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:43.934374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:43.934740 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:43.934890 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:43.935774 (+ 884us) server_negotiation.cc:299] Negotiation successful
0808 02:44:43.936009 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:44:43.939577 5571 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:42821
I20260808 02:44:43.939620 5783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:42821 every 8 connection(s)
I20260808 02:44:43.941110 5571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:44:43.943095 5785 heartbeater.cc:347] Connected to a master server at 127.25.137.254:36945
I20260808 02:44:43.943310 5785 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:43.943933 5785 heartbeater.cc:511] Master 127.25.137.254:36945 requested a full tablet report, sending...
I20260808 02:44:43.945926 5457 ts_manager.cc:194] Registered new tserver with Master: 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821)
I20260808 02:44:43.946463 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 5571
I20260808 02:44:43.946663 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:44:43.947311 26151 external_mini_cluster.cc:1428] Setting key c8b6e4516bb1a4408b4f0f49113f960e
I20260808 02:44:43.947453 5457 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:48363
WARNING: no policy specified for oryx/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab.
Entry for principal oryx/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab.
I20260808 02:44:44.001375 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:36945
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab
--principal=oryx/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:44.331555 5794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:44.331842 5794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:44.331943 5794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:44.339160 5794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:44.339268 5794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:44.339352 5794 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:44:44.339432 5794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:44:44.349268 5794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab
--principal=oryx/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.137.254:36945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:44.350605 5794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:44.352017 5794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:44.364902 5801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:44.364926 5804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:44.364859 5800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:44.366071 5794 server_base.cc:1034] running on GCE node
I20260808 02:44:44.366690 5794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:44.367669 5794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:44.368881 5794 hybrid_clock.cc:648] HybridClock initialized: now 1786157084368801 us; error 48 us; skew 500 ppm
I20260808 02:44:44.369191 5794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:44 dist-test-slave-k5rr krb5kdc[5379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157084, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:44.372262 5794 init.cc:377] Logged in from keytab as oryx/127.25.137.194@KRBTEST.COM (short username oryx)
I20260808 02:44:44.373731 5794 webserver.cc:460] Webserver started at http://127.25.137.194:44483/ using document root <none> and password file <none>
I20260808 02:44:44.374382 5794 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:44.374478 5794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:44.374749 5794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:44.377635 5794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "47488393290b420cbada0cc0bece53f1"
format_stamp: "Formatted at 2026-08-08 02:44:44 on dist-test-slave-k5rr"
server_key: "6c22d9a95b20fe32f9e7d27d741074f2"
server_key_iv: "ed7c68efc2a2afbf04ee8cc3019a7e7c"
server_key_version: "encryptionkey@0"
I20260808 02:44:44.378367 5794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "47488393290b420cbada0cc0bece53f1"
format_stamp: "Formatted at 2026-08-08 02:44:44 on dist-test-slave-k5rr"
server_key: "6c22d9a95b20fe32f9e7d27d741074f2"
server_key_iv: "ed7c68efc2a2afbf04ee8cc3019a7e7c"
server_key_version: "encryptionkey@0"
I20260808 02:44:44.383395 5794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:44:44.386703 5813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:44.387986 5794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:44:44.388137 5794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "47488393290b420cbada0cc0bece53f1"
format_stamp: "Formatted at 2026-08-08 02:44:44 on dist-test-slave-k5rr"
server_key: "6c22d9a95b20fe32f9e7d27d741074f2"
server_key_iv: "ed7c68efc2a2afbf04ee8cc3019a7e7c"
server_key_version: "encryptionkey@0"
I20260808 02:44:44.388345 5794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:44.407578 5794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:44.411744 5794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:44.411985 5794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:44.414618 5794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:44.414701 5794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:44.414824 5794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:44.414889 5794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:44:44 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157084, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:44.438493 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:44.415027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:51485 (local address 127.25.137.254:36945)
0808 02:44:44.415257 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:44.415265 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:44.416685 (+ 1420us) server_negotiation.cc:407] Connection header received
0808 02:44:44.419610 (+ 2925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:44.419618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:44.419719 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:44.419844 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:44.422391 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.423439 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.425935 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.426431 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.426560 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:44.430958 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:44.430966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:44.430971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:44.431015 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:44.434610 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:44.435788 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:44.435794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:44.435797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:44.435937 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:44.436601 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:44.436607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:44.436610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:44.436943 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:44.437116 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:44.438042 (+ 926us) server_negotiation.cc:299] Negotiation successful
0808 02:44:44.438226 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:44:44.440165 5827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:44.415599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:36945 (local address 127.25.137.194:51485)
0808 02:44:44.416503 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:44.416555 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:44.419242 (+ 2687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:44.420260 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:44.420273 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:44.421200 (+ 927us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:44.422094 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:44.422125 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.423686 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.423692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:44.425738 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:44.425769 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.426698 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.426706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:44.426938 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:44.427665 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:44.427689 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:44.430661 (+ 2972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:44.434802 (+ 4141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:44.434815 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:44.434833 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:44.435605 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:44.436093 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:44.436107 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:44.436111 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:44.436448 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:44.437087 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:44.437096 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:44.437619 (+ 523us) client_negotiation.cc:769] Sending connection context
0808 02:44:44.437939 (+ 320us) client_negotiation.cc:240] Negotiation successful
0808 02:44:44.438564 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":390,"threads_started":1}
I20260808 02:44:44.462558 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:44.447557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:36945 (local address 127.25.137.194:40179)
0808 02:44:44.448521 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:44.448538 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:44.448662 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:44.448992 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:44.448995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:44.449727 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:44.450214 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:44.450229 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.451599 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.451604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:44.452620 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:44.452633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.453565 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.453573 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:44.453764 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:44.454426 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:44.454449 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:44.455581 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:44.459448 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:44.459454 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:44.459458 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:44.460108 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:44.460568 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:44.460572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:44.460574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:44.460714 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:44.461457 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:44.461461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:44.461667 (+ 206us) client_negotiation.cc:769] Sending connection context
0808 02:44:44.461869 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 02:44:44.462187 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":715,"threads_started":1}
I20260808 02:44:44.462633 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:44.448044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:40179 (local address 127.25.137.254:36945)
0808 02:44:44.448219 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:44.448223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:44.448615 (+ 392us) server_negotiation.cc:407] Connection header received
0808 02:44:44.448776 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:44.448782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:44.448873 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:44.449037 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:44.450370 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.451457 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.452770 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.453353 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.453487 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:44.455786 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:44.455832 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:44.455835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:44.455880 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:44.459297 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:44.460251 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:44.460258 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:44.460261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:44.460444 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:44.460844 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:44.460850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:44.460853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:44.461329 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:44.461471 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:44.461997 (+ 526us) server_negotiation.cc:299] Negotiation successful
0808 02:44:44.462253 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:44:44.471117 5794 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:42207
I20260808 02:44:44.471140 5995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:42207 every 8 connection(s)
I20260808 02:44:44.472281 5794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:44:44.474351 5996 heartbeater.cc:347] Connected to a master server at 127.25.137.254:36945
I20260808 02:44:44.474563 5996 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:44.475186 5996 heartbeater.cc:511] Master 127.25.137.254:36945 requested a full tablet report, sending...
I20260808 02:44:44.476434 5457 ts_manager.cc:194] Registered new tserver with Master: 47488393290b420cbada0cc0bece53f1 (127.25.137.194:42207)
I20260808 02:44:44.477149 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 5794
I20260808 02:44:44.477330 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:44:44.477345 5457 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:51485
I20260808 02:44:44.477792 26151 external_mini_cluster.cc:1428] Setting key 4608f383710ad418d3cdf8575e3a5ed8
WARNING: no policy specified for oryx/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab.
Entry for principal oryx/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab.
I20260808 02:44:44.533999 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:36945
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab
--principal=oryx/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:44.859601 6006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:44.859860 6006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:44.859998 6006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:44.867249 6006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:44.867372 6006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:44.867503 6006 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:44:44.867619 6006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:44:44.877858 6006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab
--principal=oryx/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.137.254:36945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:44.879180 6006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:44.880820 6006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:44.893440 6013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:44.893530 6014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:44.893666 6016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:44.894721 6006 server_base.cc:1034] running on GCE node
I20260808 02:44:44.895331 6006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:44.896186 6006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:44.897408 6006 hybrid_clock.cc:648] HybridClock initialized: now 1786157084897335 us; error 47 us; skew 500 ppm
I20260808 02:44:44.897687 6006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:44 dist-test-slave-k5rr krb5kdc[5379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157084, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:44.900722 6006 init.cc:377] Logged in from keytab as oryx/127.25.137.195@KRBTEST.COM (short username oryx)
I20260808 02:44:44.902190 6006 webserver.cc:460] Webserver started at http://127.25.137.195:35225/ using document root <none> and password file <none>
I20260808 02:44:44.902841 6006 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:44.902940 6006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:44.903244 6006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:44.906132 6006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "be7a04ccfc364a55844a3b73a54a44e4"
format_stamp: "Formatted at 2026-08-08 02:44:44 on dist-test-slave-k5rr"
server_key: "0b267a249b07f84d5ffc351ff833e7e9"
server_key_iv: "69125fe071df9b5826eb8dc07c73490a"
server_key_version: "encryptionkey@0"
I20260808 02:44:44.906911 6006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "be7a04ccfc364a55844a3b73a54a44e4"
format_stamp: "Formatted at 2026-08-08 02:44:44 on dist-test-slave-k5rr"
server_key: "0b267a249b07f84d5ffc351ff833e7e9"
server_key_iv: "69125fe071df9b5826eb8dc07c73490a"
server_key_version: "encryptionkey@0"
I20260808 02:44:44.911839 6006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:44:44.915354 6026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:44.916649 6006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:44:44.916836 6006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "be7a04ccfc364a55844a3b73a54a44e4"
format_stamp: "Formatted at 2026-08-08 02:44:44 on dist-test-slave-k5rr"
server_key: "0b267a249b07f84d5ffc351ff833e7e9"
server_key_iv: "69125fe071df9b5826eb8dc07c73490a"
server_key_version: "encryptionkey@0"
I20260808 02:44:44.917071 6006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:44.933981 6006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:44.938114 6006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:44.938385 6006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:44.941866 6006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:44.941977 6006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:44.942101 6006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:44.942170 6006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:44.950241 5785 heartbeater.cc:503] Master 127.25.137.254:36945 was elected leader, sending a full tablet report...
Aug 08 02:44:44 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157084, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:44.964382 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:44.942108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:43313 (local address 127.25.137.254:36945)
0808 02:44:44.942319 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:44.942326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:44.943838 (+ 1512us) server_negotiation.cc:407] Connection header received
0808 02:44:44.946089 (+ 2251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:44.946095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:44.946184 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:44.946288 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:44.948464 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.949425 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.952092 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.952540 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.952669 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:44.957337 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:44.957345 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:44.957350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:44.957392 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:44.960752 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:44.961772 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:44.961778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:44.961781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:44.961949 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:44.962493 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:44.962497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:44.962501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:44.962825 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:44.962966 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:44.963790 (+ 824us) server_negotiation.cc:299] Negotiation successful
0808 02:44:44.964025 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:44:44.965695 6041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:44.942642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:36945 (local address 127.25.137.195:43313)
0808 02:44:44.943676 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:44.943719 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:44.945818 (+ 2099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:44.946551 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:44.946560 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:44.947270 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:44.948182 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:44.948215 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.949596 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.949602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:44.951944 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:44.951963 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.952803 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.952811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:44.953167 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:44.954012 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:44.954058 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:44.957046 (+ 2988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:44.960890 (+ 3844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:44.960901 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:44.960919 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:44.961640 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:44.962072 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:44.962078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:44.962082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:44.962322 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:44.963041 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:44.963050 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:44.963405 (+ 355us) client_negotiation.cc:769] Sending connection context
0808 02:44:44.963630 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 02:44:44.964171 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":434,"threads_started":1}
I20260808 02:44:44.986163 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:44.971163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:39953 (local address 127.25.137.254:36945)
0808 02:44:44.971355 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:44.971361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:44.971977 (+ 616us) server_negotiation.cc:407] Connection header received
0808 02:44:44.972162 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:44.972166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:44.972255 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:44.972355 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:44.973864 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.974835 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.976827 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.977313 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.977445 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:44.979871 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:44.979878 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:44.979882 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:44.979924 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:44.983131 (+ 3207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:44.983977 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:44.983983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:44.983986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:44.984141 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:44.984570 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:44.984576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:44.984578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:44.984948 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:44.985119 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:44.985584 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 02:44:44.985812 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:44:44.986256 6130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:44.971163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:36945 (local address 127.25.137.195:39953)
0808 02:44:44.971867 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:44.971889 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:44.972041 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:44.972417 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:44.972422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:44.973088 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:44.973666 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:44.973685 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.974998 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.975003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:44.976679 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:44.976695 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:44.977510 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:44.977517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:44.977743 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:44.978468 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:44.978497 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:44.979666 (+ 1169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:44.983269 (+ 3603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:44.983279 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:44.983283 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:44.983844 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:44.984243 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:44.984249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:44.984253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:44.984403 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:44.985096 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:44.985101 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:44.985318 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 02:44:44.985465 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 02:44:44.985804 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":343,"threads_started":1}
I20260808 02:44:44.994589 6006 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:46633
I20260808 02:44:44.994618 6216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:46633 every 8 connection(s)
I20260808 02:44:44.995795 6006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:44:44.997771 6217 heartbeater.cc:347] Connected to a master server at 127.25.137.254:36945
I20260808 02:44:44.997992 6217 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:44.998570 6217 heartbeater.cc:511] Master 127.25.137.254:36945 requested a full tablet report, sending...
I20260808 02:44:44.999645 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 6006
I20260808 02:44:44.999819 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:44:44.999900 5457 ts_manager.cc:194] Registered new tserver with Master: be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633)
I20260808 02:44:45.000276 26151 external_mini_cluster.cc:1428] Setting key 210c500eb12dd26775d61f35d219cdc3
I20260808 02:44:45.000845 5457 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:43313
I20260808 02:44:45.003177 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:44:45 dist-test-slave-k5rr krb5kdc[5379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157085, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:44:45 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:45 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:44:45.060823 5609 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:45.050461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49084 (local address 127.25.137.254:36945)
0808 02:44:45.050672 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.050677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.051080 (+ 403us) server_negotiation.cc:407] Connection header received
0808 02:44:45.051235 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.051240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.051324 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.051413 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:45.052721 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.053504 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.054855 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.055283 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.055388 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.060531 (+ 5143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49084: BlockingRecv error: recv got EOF from 127.0.0.1:49084 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 08 02:44:45 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157085, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:45.077214 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.064019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.25.137.254:36945)
0808 02:44:45.064185 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.064189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.064564 (+ 375us) server_negotiation.cc:407] Connection header received
0808 02:44:45.064668 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.064672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.064751 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.064838 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:45.065981 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.066726 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.067997 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.068470 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.068579 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.072158 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:45.072163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:45.072166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:45.072199 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:45.074751 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:45.075377 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:45.075381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:45.075384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:45.075504 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:45.075875 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:45.075880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:45.075883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:45.076150 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:45.076432 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.076673 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.076901 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:44:45.081453 5456 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:44:45.088322 5456 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:44:45.111670 6247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.100543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:46633 (local address 127.0.0.1:57032)
0808 02:44:45.101771 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:45.101829 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:45.102001 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:45.103872 (+ 1871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:45.103878 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:45.103914 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:45.104457 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.104479 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.106817 (+ 2338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.106821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.108764 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.108780 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.110574 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.110580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.110765 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.110817 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 02:44:45.110959 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 02:44:45.111153 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":29,"spinlock_wait_cycles":3456,"thread_start_us":2169,"threads_started":1}
I20260808 02:44:45.111822 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.100888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:42207 (local address 127.0.0.1:48716)
0808 02:44:45.103004 (+ 2116us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:45.103022 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:45.103144 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:45.104050 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:45.104056 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:45.104088 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:45.104786 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.104812 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.106642 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.106647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.109191 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.109207 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.110917 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.110923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.111089 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.111117 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 02:44:45.111222 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 02:44:45.111362 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1988,"mutex_wait_us":30,"thread_start_us":774,"threads_started":1}
I20260808 02:44:45.112596 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.101481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48716 (local address 127.25.137.194:42207)
0808 02:44:45.102655 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.102664 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.103119 (+ 455us) server_negotiation.cc:407] Connection header received
0808 02:44:45.103347 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.103355 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.103591 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.103813 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:45.104980 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.106493 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.109361 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.110685 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.110863 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.111019 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.112156 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.112344 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":903,"threads_started":1}
I20260808 02:44:45.112668 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.100923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57032 (local address 127.25.137.195:46633)
0808 02:44:45.103190 (+ 2267us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.103198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.103232 (+ 34us) server_negotiation.cc:407] Connection header received
0808 02:44:45.103333 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.103340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.103609 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.104749 (+ 1140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:45.105122 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.106688 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.108944 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.110369 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.110570 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.110751 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.112147 (+ 1396us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.112336 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2101,"thread_start_us":1916,"threads_started":1}
I20260808 02:44:45.113189 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.103731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42821 (local address 127.0.0.1:48832)
0808 02:44:45.105304 (+ 1573us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:45.105318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:45.105428 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:45.105899 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:45.105903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:45.105926 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:45.106235 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.106244 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.108034 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.108040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.109830 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.109842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.111729 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.111735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.112315 (+ 580us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.112347 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:44:45.112445 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 02:44:45.112565 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1466,"thread_start_us":1332,"threads_started":1}
I20260808 02:44:45.113703 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.103424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48832 (local address 127.25.137.193:42821)
0808 02:44:45.104827 (+ 1403us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.104835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.105414 (+ 579us) server_negotiation.cc:407] Connection header received
0808 02:44:45.105559 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.105566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.105736 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.105896 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:45.106393 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.107845 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.109981 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.111230 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.111399 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.111568 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.112509 (+ 941us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.112681 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1237,"thread_start_us":1099,"threads_started":1}
I20260808 02:44:45.116386 5894 tablet_service.cc:1467] Processing CreateTablet for tablet 7aec632747b04b7594b2c38b22cd629d (DEFAULT_TABLE table=test-table [id=b0046d5411604322b469898c623098b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:45.116386 6111 tablet_service.cc:1467] Processing CreateTablet for tablet 7aec632747b04b7594b2c38b22cd629d (DEFAULT_TABLE table=test-table [id=b0046d5411604322b469898c623098b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:45.117882 5894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aec632747b04b7594b2c38b22cd629d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:45.117880 6111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aec632747b04b7594b2c38b22cd629d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:45.118839 5677 tablet_service.cc:1467] Processing CreateTablet for tablet 7aec632747b04b7594b2c38b22cd629d (DEFAULT_TABLE table=test-table [id=b0046d5411604322b469898c623098b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:45.120029 5677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aec632747b04b7594b2c38b22cd629d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:45.133778 6258 tablet_bootstrap.cc:492] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: Bootstrap starting.
I20260808 02:44:45.134877 6260 tablet_bootstrap.cc:492] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4: Bootstrap starting.
I20260808 02:44:45.137704 6263 tablet_bootstrap.cc:492] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: Bootstrap starting.
I20260808 02:44:45.138643 6260 tablet_bootstrap.cc:654] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:45.139437 6258 tablet_bootstrap.cc:654] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:45.139983 6260 log.cc:824] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:45.140849 6258 log.cc:824] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:45.140866 6263 tablet_bootstrap.cc:654] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:45.142086 6263 log.cc:824] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:45.143246 6260 tablet_bootstrap.cc:492] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4: No bootstrap required, opened a new log
I20260808 02:44:45.143525 6260 ts_tablet_manager.cc:1397] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 02:44:45.143762 6258 tablet_bootstrap.cc:492] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: No bootstrap required, opened a new log
I20260808 02:44:45.144012 6258 ts_tablet_manager.cc:1397] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260808 02:44:45.145103 6263 tablet_bootstrap.cc:492] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: No bootstrap required, opened a new log
I20260808 02:44:45.145375 6263 ts_tablet_manager.cc:1397] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 02:44:45.151719 6258 raft_consensus.cc:348] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } }
I20260808 02:44:45.152099 6258 raft_consensus.cc:374] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:45.152199 6258 raft_consensus.cc:729] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afcd23c51b249c3912614f4d62f07d4, State: Initialized, Role: FOLLOWER
I20260808 02:44:45.152786 6258 consensus_queue.cc:260] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } }
I20260808 02:44:45.154554 6258 ts_tablet_manager.cc:1428] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260808 02:44:45.154417 6260 raft_consensus.cc:348] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } }
I20260808 02:44:45.154841 6260 raft_consensus.cc:374] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:45.154944 6260 raft_consensus.cc:729] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be7a04ccfc364a55844a3b73a54a44e4, State: Initialized, Role: FOLLOWER
I20260808 02:44:45.155543 6260 consensus_queue.cc:260] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } }
I20260808 02:44:45.155774 6263 raft_consensus.cc:348] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } }
I20260808 02:44:45.156196 6263 raft_consensus.cc:374] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:45.156315 6263 raft_consensus.cc:729] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47488393290b420cbada0cc0bece53f1, State: Initialized, Role: FOLLOWER
I20260808 02:44:45.156973 6217 heartbeater.cc:503] Master 127.25.137.254:36945 was elected leader, sending a full tablet report...
I20260808 02:44:45.157626 6260 ts_tablet_manager.cc:1428] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260808 02:44:45.158414 6263 consensus_queue.cc:260] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } }
I20260808 02:44:45.158914 6272 raft_consensus.cc:482] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:45.159521 5996 heartbeater.cc:503] Master 127.25.137.254:36945 was elected leader, sending a full tablet report...
I20260808 02:44:45.159206 6272 raft_consensus.cc:504] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } }
I20260808 02:44:45.163542 6272 leader_election.cc:288] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633), 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821)
I20260808 02:44:45.164800 6263 ts_tablet_manager.cc:1428] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: Time spent starting tablet: real 0.019s user 0.007s sys 0.005s
I20260808 02:44:45.172238 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.163976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:33385 (local address 127.25.137.195:46633)
0808 02:44:45.164175 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.164182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.164748 (+ 566us) server_negotiation.cc:407] Connection header received
0808 02:44:45.164983 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.164988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.165105 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.165184 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:45.167138 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.168030 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.170491 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.171260 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.171434 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.171523 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.171855 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.171984 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:44:45.172415 6276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.163958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:46633 (local address 127.25.137.194:33385)
0808 02:44:45.164645 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:45.164662 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:45.164807 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:45.165429 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:45.165435 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:45.165464 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:45.166897 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.166918 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.168159 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.168165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.170344 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.170360 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.171520 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.171526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.171703 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.171731 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 02:44:45.171827 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 02:44:45.171977 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":1,"thread_start_us":356,"threads_started":1}
I20260808 02:44:45.173524 6150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aec632747b04b7594b2c38b22cd629d" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be7a04ccfc364a55844a3b73a54a44e4" is_pre_election: true
I20260808 02:44:45.174198 6150 raft_consensus.cc:2454] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47488393290b420cbada0cc0bece53f1 in term 0.
I20260808 02:44:45.174922 5817 leader_election.cc:302] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47488393290b420cbada0cc0bece53f1, be7a04ccfc364a55844a3b73a54a44e4; no voters:
I20260808 02:44:45.175136 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.165117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:36665 (local address 127.25.137.193:42821)
0808 02:44:45.165341 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.165347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.169521 (+ 4174us) server_negotiation.cc:407] Connection header received
0808 02:44:45.169634 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.169640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.169713 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.169795 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:45.170333 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.171042 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.172900 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.174118 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.174283 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.174368 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.174769 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.174909 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:44:45.175180 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.164774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42821 (local address 127.25.137.194:36665)
0808 02:44:45.169417 (+ 4643us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:45.169432 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:45.169541 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:45.169813 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:45.169816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:45.169832 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:45.170179 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.170187 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.171173 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.171179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.172742 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.172756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.174322 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.174345 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.174585 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.174620 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:44:45.174702 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 02:44:45.174812 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4534,"thread_start_us":4399,"threads_started":1}
I20260808 02:44:45.175393 6272 raft_consensus.cc:2781] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:44:45.175513 6272 raft_consensus.cc:482] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:44:45.175585 6272 raft_consensus.cc:3037] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:45.177750 6272 raft_consensus.cc:504] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } }
I20260808 02:44:45.178365 6272 leader_election.cc:288] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 election: Requested vote from peers be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633), 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821)
I20260808 02:44:45.179116 6150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aec632747b04b7594b2c38b22cd629d" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be7a04ccfc364a55844a3b73a54a44e4"
I20260808 02:44:45.179337 6150 raft_consensus.cc:3037] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:45.179577 5702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aec632747b04b7594b2c38b22cd629d" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcd23c51b249c3912614f4d62f07d4"
I20260808 02:44:45.179649 5704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aec632747b04b7594b2c38b22cd629d" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcd23c51b249c3912614f4d62f07d4" is_pre_election: true
I20260808 02:44:45.180079 5702 raft_consensus.cc:3037] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:45.182482 6150 raft_consensus.cc:2454] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47488393290b420cbada0cc0bece53f1 in term 1.
I20260808 02:44:45.182837 5817 leader_election.cc:302] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47488393290b420cbada0cc0bece53f1, be7a04ccfc364a55844a3b73a54a44e4; no voters:
I20260808 02:44:45.183125 6272 raft_consensus.cc:2781] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:45.183236 5702 raft_consensus.cc:2454] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47488393290b420cbada0cc0bece53f1 in term 1.
I20260808 02:44:45.183308 6272 raft_consensus.cc:686] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 1 LEADER]: Becoming Leader. State: Replica: 47488393290b420cbada0cc0bece53f1, State: Running, Role: LEADER
I20260808 02:44:45.183825 6272 consensus_queue.cc:237] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } }
I20260808 02:44:45.188060 5457 catalog_manager.cc:5495] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47488393290b420cbada0cc0bece53f1 (127.25.137.194). New cstate: current_term: 1 leader_uuid: "47488393290b420cbada0cc0bece53f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } health_report { overall_health: HEALTHY } } }
W20260808 02:44:45.193388 5787 tablet.cc:2369] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:44:45.209860 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.205110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48732 (local address 127.25.137.194:42207)
0808 02:44:45.205279 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.205283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.205345 (+ 62us) server_negotiation.cc:407] Connection header received
0808 02:44:45.205530 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.205533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.205620 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.205739 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:45.206026 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.206746 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.207967 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.208367 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.208454 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.208794 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:45.209416 (+ 622us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:45.209522 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.209586 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.209652 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:44:45.217717 5702 raft_consensus.cc:1261] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Refusing update from remote peer 47488393290b420cbada0cc0bece53f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:45.217905 6150 raft_consensus.cc:1261] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Refusing update from remote peer 47488393290b420cbada0cc0bece53f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:45.218534 6275 consensus_queue.cc:1035] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:45.218919 6272 consensus_queue.cc:1035] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:44:45.224231 5999 tablet.cc:2369] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:44:45.230324 6280 mvcc.cc:205] Tried to move back new op lower bound from 7316099421042634752 to 7316099420917227520. Current Snapshot: MvccSnapshot[applied={T|T < 7316099421042634752}]
I20260808 02:44:45.247174 5456 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:45.247495 5456 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:45.251437 5456 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5: Sending DeleteTablet for 3 replicas of tablet 7aec632747b04b7594b2c38b22cd629d
I20260808 02:44:45.252852 5894 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aec632747b04b7594b2c38b22cd629d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:45 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:48716
I20260808 02:44:45.253103 6111 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aec632747b04b7594b2c38b22cd629d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:45 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:57032
I20260808 02:44:45.253103 5677 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aec632747b04b7594b2c38b22cd629d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:45 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:48832
I20260808 02:44:45.254871 5457 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:44:45.255656 5457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:44:45.264587 5892 tablet_service.cc:1467] Processing CreateTablet for tablet 11b3bda6a79c443b9d025ef6c0d6325a (DEFAULT_TABLE table=test-table [id=4452d9d295c042b4a6867d39d3fc88ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:45.265354 5892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11b3bda6a79c443b9d025ef6c0d6325a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:45.266840 6305 tablet_replica.cc:330] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: stopping tablet replica
I20260808 02:44:45.267567 6305 raft_consensus.cc:2229] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:45.268515 5677 tablet_service.cc:1467] Processing CreateTablet for tablet 11b3bda6a79c443b9d025ef6c0d6325a (DEFAULT_TABLE table=test-table [id=4452d9d295c042b4a6867d39d3fc88ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:45.269132 5677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11b3bda6a79c443b9d025ef6c0d6325a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:45.269078 6109 tablet_service.cc:1467] Processing CreateTablet for tablet 11b3bda6a79c443b9d025ef6c0d6325a (DEFAULT_TABLE table=test-table [id=4452d9d295c042b4a6867d39d3fc88ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:45.269766 6109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11b3bda6a79c443b9d025ef6c0d6325a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:45.272260 6302 tablet_replica.cc:330] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4: stopping tablet replica
I20260808 02:44:45.272751 6302 raft_consensus.cc:2229] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:45.273437 6302 raft_consensus.cc:2258] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:45.273751 6305 raft_consensus.cc:2258] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:45.275581 6302 ts_tablet_manager.cc:1905] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:45.276238 6263 tablet_bootstrap.cc:492] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1: Bootstrap starting.
I20260808 02:44:45.276714 6260 tablet_bootstrap.cc:492] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4: Bootstrap starting.
I20260808 02:44:45.277793 6305 ts_tablet_manager.cc:1905] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:45.278049 6263 tablet_bootstrap.cc:654] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:45.278237 6260 tablet_bootstrap.cc:654] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:45.279768 6258 tablet_bootstrap.cc:492] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4: Bootstrap starting.
I20260808 02:44:45.281756 6258 tablet_bootstrap.cc:654] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:45.292281 6302 ts_tablet_manager.cc:1918] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:45.292469 6302 log.cc:1196] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/7aec632747b04b7594b2c38b22cd629d
I20260808 02:44:45.296440 6305 ts_tablet_manager.cc:1918] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:45.297847 6263 tablet_bootstrap.cc:492] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1: No bootstrap required, opened a new log
I20260808 02:44:45.298025 6263 ts_tablet_manager.cc:1397] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.000s
I20260808 02:44:45.298511 6258 tablet_bootstrap.cc:492] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4: No bootstrap required, opened a new log
I20260808 02:44:45.298810 6258 ts_tablet_manager.cc:1397] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.000s
I20260808 02:44:45.298943 6263 raft_consensus.cc:348] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.299203 6263 raft_consensus.cc:374] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:45.299340 6263 raft_consensus.cc:729] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47488393290b420cbada0cc0bece53f1, State: Initialized, Role: FOLLOWER
I20260808 02:44:45.299620 6263 consensus_queue.cc:260] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.299994 6258 raft_consensus.cc:348] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.300299 6258 raft_consensus.cc:374] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:45.300449 6258 raft_consensus.cc:729] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afcd23c51b249c3912614f4d62f07d4, State: Initialized, Role: FOLLOWER
I20260808 02:44:45.300814 6258 consensus_queue.cc:260] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.302083 6260 tablet_bootstrap.cc:492] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4: No bootstrap required, opened a new log
I20260808 02:44:45.302219 6260 ts_tablet_manager.cc:1397] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.003s
I20260808 02:44:45.302930 6260 raft_consensus.cc:348] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.303133 6260 raft_consensus.cc:374] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:45.303206 6260 raft_consensus.cc:729] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be7a04ccfc364a55844a3b73a54a44e4, State: Initialized, Role: FOLLOWER
I20260808 02:44:45.303449 6260 consensus_queue.cc:260] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.304313 6258 ts_tablet_manager.cc:1428] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260808 02:44:45.305423 6305 log.cc:1196] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/7aec632747b04b7594b2c38b22cd629d
I20260808 02:44:45.306056 6305 ts_tablet_manager.cc:1939] T 7aec632747b04b7594b2c38b22cd629d P 8afcd23c51b249c3912614f4d62f07d4: Deleting consensus metadata
I20260808 02:44:45.307938 5429 catalog_manager.cc:4841] TS 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821): tablet 7aec632747b04b7594b2c38b22cd629d (table test-table [id=b0046d5411604322b469898c623098b5]) successfully deleted
I20260808 02:44:45.308498 6260 ts_tablet_manager.cc:1428] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260808 02:44:45.309535 6301 tablet_replica.cc:330] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: stopping tablet replica
I20260808 02:44:45.309990 6301 raft_consensus.cc:2229] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:44:45.310664 6301 raft_consensus.cc:2258] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:45.311360 6263 ts_tablet_manager.cc:1428] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1: Time spent starting tablet: real 0.013s user 0.003s sys 0.000s
I20260808 02:44:45.312788 6301 ts_tablet_manager.cc:1905] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:45.314445 6302 ts_tablet_manager.cc:1939] T 7aec632747b04b7594b2c38b22cd629d P be7a04ccfc364a55844a3b73a54a44e4: Deleting consensus metadata
I20260808 02:44:45.316119 5429 catalog_manager.cc:4841] TS be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633): tablet 7aec632747b04b7594b2c38b22cd629d (table test-table [id=b0046d5411604322b469898c623098b5]) successfully deleted
I20260808 02:44:45.320214 6301 ts_tablet_manager.cc:1918] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:45.320442 6301 log.cc:1196] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/7aec632747b04b7594b2c38b22cd629d
I20260808 02:44:45.321211 6301 ts_tablet_manager.cc:1939] T 7aec632747b04b7594b2c38b22cd629d P 47488393290b420cbada0cc0bece53f1: Deleting consensus metadata
I20260808 02:44:45.322997 5431 catalog_manager.cc:4841] TS 47488393290b420cbada0cc0bece53f1 (127.25.137.194:42207): tablet 7aec632747b04b7594b2c38b22cd629d (table test-table [id=b0046d5411604322b469898c623098b5]) successfully deleted
W20260808 02:44:45.498459 6220 tablet.cc:2369] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:44:45.615803 6270 raft_consensus.cc:482] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:45.616077 6270 raft_consensus.cc:504] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.617856 6270 leader_election.cc:288] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47488393290b420cbada0cc0bece53f1 (127.25.137.194:42207), be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633)
I20260808 02:44:45.626683 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.618177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:42207 (local address 127.25.137.193:57781)
0808 02:44:45.619398 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:45.619421 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:45.619666 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:45.620170 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:45.620175 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:45.620200 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:45.620880 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.620895 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.622501 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.622507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.624477 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.624488 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.625770 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.625776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.625955 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.625990 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:44:45.626136 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 02:44:45.626282 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1080,"mutex_wait_us":34,"spinlock_wait_cycles":7936,"thread_start_us":1246,"threads_started":1}
I20260808 02:44:45.626760 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.618874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:57781 (local address 127.25.137.194:42207)
0808 02:44:45.619031 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.619037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.619531 (+ 494us) server_negotiation.cc:407] Connection header received
0808 02:44:45.619876 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.619883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.620012 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.620139 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:45.621115 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.622355 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.624612 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.625564 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.625711 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.625794 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.626195 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.626414 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:44:45.627089 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.618482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:49871 (local address 127.25.137.195:46633)
0808 02:44:45.618672 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.618679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.619839 (+ 1160us) server_negotiation.cc:407] Connection header received
0808 02:44:45.619990 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.619994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.620061 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.620133 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:45.621157 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.622049 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.624554 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.625803 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.625975 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.626057 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.626393 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.626531 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:44:45.627765 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.618365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:46633 (local address 127.25.137.193:49871)
0808 02:44:45.619746 (+ 1381us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:45.619763 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:45.619864 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:45.620302 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:45.620306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:45.620337 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:45.620891 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.620901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.622170 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.622176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.624365 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.624385 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.626018 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.626024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.626212 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.626243 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:44:45.627379 (+ 1136us) client_negotiation.cc:240] Negotiation successful
0808 02:44:45.627486 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1255,"mutex_wait_us":41,"spinlock_wait_cycles":896,"thread_start_us":707,"threads_started":1}
I20260808 02:44:45.627863 5931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b3bda6a79c443b9d025ef6c0d6325a" candidate_uuid: "8afcd23c51b249c3912614f4d62f07d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47488393290b420cbada0cc0bece53f1" is_pre_election: true
I20260808 02:44:45.628409 6150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b3bda6a79c443b9d025ef6c0d6325a" candidate_uuid: "8afcd23c51b249c3912614f4d62f07d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be7a04ccfc364a55844a3b73a54a44e4" is_pre_election: true
I20260808 02:44:45.628454 5931 raft_consensus.cc:2454] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8afcd23c51b249c3912614f4d62f07d4 in term 0.
I20260808 02:44:45.628623 6150 raft_consensus.cc:2454] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8afcd23c51b249c3912614f4d62f07d4 in term 0.
I20260808 02:44:45.629199 5596 leader_election.cc:302] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47488393290b420cbada0cc0bece53f1, 8afcd23c51b249c3912614f4d62f07d4; no voters:
I20260808 02:44:45.629664 6270 raft_consensus.cc:2781] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:44:45.629805 6270 raft_consensus.cc:482] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:44:45.629908 6270 raft_consensus.cc:3037] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:45.632086 6270 raft_consensus.cc:504] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.632678 6270 leader_election.cc:288] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [CANDIDATE]: Term 1 election: Requested vote from peers 47488393290b420cbada0cc0bece53f1 (127.25.137.194:42207), be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633)
I20260808 02:44:45.633205 6150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b3bda6a79c443b9d025ef6c0d6325a" candidate_uuid: "8afcd23c51b249c3912614f4d62f07d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be7a04ccfc364a55844a3b73a54a44e4"
I20260808 02:44:45.633193 5931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b3bda6a79c443b9d025ef6c0d6325a" candidate_uuid: "8afcd23c51b249c3912614f4d62f07d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47488393290b420cbada0cc0bece53f1"
I20260808 02:44:45.633399 6150 raft_consensus.cc:3037] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:45.633405 5931 raft_consensus.cc:3037] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:45.636225 6150 raft_consensus.cc:2454] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8afcd23c51b249c3912614f4d62f07d4 in term 1.
I20260808 02:44:45.636663 5931 raft_consensus.cc:2454] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8afcd23c51b249c3912614f4d62f07d4 in term 1.
I20260808 02:44:45.636686 5595 leader_election.cc:302] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8afcd23c51b249c3912614f4d62f07d4, be7a04ccfc364a55844a3b73a54a44e4; no voters:
I20260808 02:44:45.636988 6270 raft_consensus.cc:2781] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:45.637506 6270 raft_consensus.cc:686] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 1 LEADER]: Becoming Leader. State: Replica: 8afcd23c51b249c3912614f4d62f07d4, State: Running, Role: LEADER
I20260808 02:44:45.637908 6270 consensus_queue.cc:237] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.641482 5457 catalog_manager.cc:5495] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193). New cstate: current_term: 1 leader_uuid: "8afcd23c51b249c3912614f4d62f07d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:44:45 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157083, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.254@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:45.695617 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.675369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:36945 (local address 127.0.0.1:49092)
0808 02:44:45.676098 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:45.676123 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:45.676322 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:45.676704 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:45.676710 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:45.677618 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:45.677885 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.677899 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.679010 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.679016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.680617 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.680633 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.681416 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.681424 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.681619 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.686206 (+ 4587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:45.686246 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:45.688958 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:45.692852 (+ 3894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:45.692869 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:45.692878 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:45.693521 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:45.693916 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:45.693919 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:45.693921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:45.694067 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:45.694746 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:45.694752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:45.694961 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 02:44:45.695099 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 02:44:45.695324 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":360,"threads_started":1}
I20260808 02:44:45.695988 6327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.675391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49092 (local address 127.25.137.254:36945)
0808 02:44:45.676398 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.676401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.676422 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:44:45.676470 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.676473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.676535 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.676623 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:45.678042 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.678863 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.680775 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.681226 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.681352 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.689210 (+ 7858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:45.689218 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:45.689223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:45.689270 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:45.692620 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:45.693641 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:45.693647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:45.693651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:45.693806 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:45.694180 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:45.694185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:45.694189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:45.694595 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:45.694748 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.695282 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.695522 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":908,"thread_start_us":771,"threads_started":1}
I20260808 02:44:45.701320 5457 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:49092:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:44:45.702481 5457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:44:45.709337 5677 tablet_service.cc:1467] Processing CreateTablet for tablet ea23c6d4eb5c4ecb9b72b31267611d65 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1dcfca66e1324636824af2bbf6ef1403]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:44:45.709399 5894 tablet_service.cc:1467] Processing CreateTablet for tablet ea23c6d4eb5c4ecb9b72b31267611d65 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1dcfca66e1324636824af2bbf6ef1403]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:44:45.709650 6109 tablet_service.cc:1467] Processing CreateTablet for tablet ea23c6d4eb5c4ecb9b72b31267611d65 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1dcfca66e1324636824af2bbf6ef1403]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:44:45.709977 5677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea23c6d4eb5c4ecb9b72b31267611d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:45.710065 5894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea23c6d4eb5c4ecb9b72b31267611d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:45.710203 6109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea23c6d4eb5c4ecb9b72b31267611d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:45.715761 6258 tablet_bootstrap.cc:492] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4: Bootstrap starting.
I20260808 02:44:45.717233 6258 tablet_bootstrap.cc:654] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:45.717262 6263 tablet_bootstrap.cc:492] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: Bootstrap starting.
I20260808 02:44:45.717299 6260 tablet_bootstrap.cc:492] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4: Bootstrap starting.
I20260808 02:44:45.718976 6263 tablet_bootstrap.cc:654] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:45.719029 6260 tablet_bootstrap.cc:654] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:45.719945 6258 tablet_bootstrap.cc:492] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4: No bootstrap required, opened a new log
I20260808 02:44:45.720129 6258 ts_tablet_manager.cc:1397] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 02:44:45.720980 6258 raft_consensus.cc:348] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.721259 6258 raft_consensus.cc:374] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:45.721360 6258 raft_consensus.cc:729] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afcd23c51b249c3912614f4d62f07d4, State: Initialized, Role: FOLLOWER
I20260808 02:44:45.721463 6260 tablet_bootstrap.cc:492] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4: No bootstrap required, opened a new log
I20260808 02:44:45.721612 6260 ts_tablet_manager.cc:1397] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 02:44:45.721612 6258 consensus_queue.cc:260] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.722028 6263 tablet_bootstrap.cc:492] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: No bootstrap required, opened a new log
I20260808 02:44:45.722199 6258 ts_tablet_manager.cc:1428] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:44:45.722203 6263 ts_tablet_manager.cc:1397] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260808 02:44:45.722059 6270 tablet_replica.cc:439] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } }
I20260808 02:44:45.722357 6270 tablet_replica.cc:442] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:44:45.722862 6263 raft_consensus.cc:348] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.723142 6263 raft_consensus.cc:374] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:45.723222 6263 raft_consensus.cc:729] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47488393290b420cbada0cc0bece53f1, State: Initialized, Role: FOLLOWER
I20260808 02:44:45.723488 6260 raft_consensus.cc:348] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.723452 6263 consensus_queue.cc:260] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.723680 6260 raft_consensus.cc:374] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:45.723757 6260 raft_consensus.cc:729] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be7a04ccfc364a55844a3b73a54a44e4, State: Initialized, Role: FOLLOWER
I20260808 02:44:45.724050 6263 ts_tablet_manager.cc:1428] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 02:44:45.724013 6260 consensus_queue.cc:260] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.723946 6297 tablet_replica.cc:439] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } }
I20260808 02:44:45.724267 6297 tablet_replica.cc:442] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:44:45.724450 6271 tablet_replica.cc:439] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } }
I20260808 02:44:45.724732 6271 tablet_replica.cc:442] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:44:45.725335 6260 ts_tablet_manager.cc:1428] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260808 02:44:45.768857 6297 raft_consensus.cc:482] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:45.769086 6297 raft_consensus.cc:504] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.769768 6297 leader_election.cc:288] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821), be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633)
I20260808 02:44:45.770367 5702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea23c6d4eb5c4ecb9b72b31267611d65" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcd23c51b249c3912614f4d62f07d4" is_pre_election: true
I20260808 02:44:45.770406 6150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea23c6d4eb5c4ecb9b72b31267611d65" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be7a04ccfc364a55844a3b73a54a44e4" is_pre_election: true
I20260808 02:44:45.770620 5702 raft_consensus.cc:2454] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47488393290b420cbada0cc0bece53f1 in term 0.
I20260808 02:44:45.770658 6150 raft_consensus.cc:2454] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47488393290b420cbada0cc0bece53f1 in term 0.
I20260808 02:44:45.771056 5817 leader_election.cc:302] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47488393290b420cbada0cc0bece53f1, 8afcd23c51b249c3912614f4d62f07d4; no voters:
I20260808 02:44:45.771327 6297 raft_consensus.cc:2781] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:44:45.771468 6297 raft_consensus.cc:482] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:44:45.771587 6297 raft_consensus.cc:3037] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:45.773725 6297 raft_consensus.cc:504] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.774314 6297 leader_election.cc:288] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 election: Requested vote from peers 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821), be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633)
I20260808 02:44:45.774829 5702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea23c6d4eb5c4ecb9b72b31267611d65" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcd23c51b249c3912614f4d62f07d4"
I20260808 02:44:45.774905 6150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea23c6d4eb5c4ecb9b72b31267611d65" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be7a04ccfc364a55844a3b73a54a44e4"
I20260808 02:44:45.775069 5702 raft_consensus.cc:3037] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:45.775125 6150 raft_consensus.cc:3037] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:45.777175 6150 raft_consensus.cc:2454] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47488393290b420cbada0cc0bece53f1 in term 1.
I20260808 02:44:45.777316 5702 raft_consensus.cc:2454] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47488393290b420cbada0cc0bece53f1 in term 1.
I20260808 02:44:45.777581 5817 leader_election.cc:302] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47488393290b420cbada0cc0bece53f1, be7a04ccfc364a55844a3b73a54a44e4; no voters:
I20260808 02:44:45.777909 6297 raft_consensus.cc:2781] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:45.778358 6297 raft_consensus.cc:686] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [term 1 LEADER]: Becoming Leader. State: Replica: 47488393290b420cbada0cc0bece53f1, State: Running, Role: LEADER
I20260808 02:44:45.778642 6297 consensus_queue.cc:237] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:45.780193 6334 tablet_replica.cc:439] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: TxnStatusTablet state changed. Reason: New leader 47488393290b420cbada0cc0bece53f1. Latest consensus state: current_term: 1 leader_uuid: "47488393290b420cbada0cc0bece53f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } }
I20260808 02:44:45.780500 6334 tablet_replica.cc:442] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:44:45.781174 6336 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:44:45.781438 6336 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:44:45.781865 5453 catalog_manager.cc:5495] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47488393290b420cbada0cc0bece53f1 (127.25.137.194). New cstate: current_term: 1 leader_uuid: "47488393290b420cbada0cc0bece53f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } health_report { overall_health: UNKNOWN } } }
I20260808 02:44:45.807498 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.801862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:42207 (local address 127.0.0.1:48736)
0808 02:44:45.802091 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:45.802109 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:45.802242 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:45.802633 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:45.802636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:45.802646 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:44:45.802821 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.802829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.804246 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.804250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.805316 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.805330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.806395 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.806400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.806562 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.806618 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:44:45.807058 (+ 440us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:44:45.807070 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 02:44:45.807129 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:44:45.807246 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 02:44:45.807542 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.801928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48736 (local address 127.25.137.194:42207)
0808 02:44:45.802146 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.802153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.802175 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:44:45.802416 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.802422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.802516 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.802620 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:45.802955 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.804089 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.805455 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.806143 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.806307 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.806804 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:45.806968 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:45.807059 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.807169 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.807261 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:44:45.811545 6150 raft_consensus.cc:1261] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Refusing update from remote peer 47488393290b420cbada0cc0bece53f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:45.811625 5702 raft_consensus.cc:1261] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Refusing update from remote peer 47488393290b420cbada0cc0bece53f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:45.812163 6297 consensus_queue.cc:1035] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:45.812500 6334 consensus_queue.cc:1035] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:45.821623 6270 tablet_replica.cc:439] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4: TxnStatusTablet state changed. Reason: New leader 47488393290b420cbada0cc0bece53f1. Latest consensus state: current_term: 1 leader_uuid: "47488393290b420cbada0cc0bece53f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } }
I20260808 02:44:45.821899 6270 tablet_replica.cc:442] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:44:45.824963 6334 tablet_replica.cc:439] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "47488393290b420cbada0cc0bece53f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } }
I20260808 02:44:45.825274 6334 tablet_replica.cc:442] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:44:45.830040 6270 tablet_replica.cc:439] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "47488393290b420cbada0cc0bece53f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } }
I20260808 02:44:45.830350 6270 tablet_replica.cc:442] T ea23c6d4eb5c4ecb9b72b31267611d65 P 8afcd23c51b249c3912614f4d62f07d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:44:45.834524 6271 tablet_replica.cc:439] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4: TxnStatusTablet state changed. Reason: New leader 47488393290b420cbada0cc0bece53f1. Latest consensus state: current_term: 1 leader_uuid: "47488393290b420cbada0cc0bece53f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } }
I20260808 02:44:45.834836 6271 tablet_replica.cc:442] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:44:45.836193 6297 tablet_replica.cc:439] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "47488393290b420cbada0cc0bece53f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } }
I20260808 02:44:45.836516 6297 tablet_replica.cc:442] T ea23c6d4eb5c4ecb9b72b31267611d65 P 47488393290b420cbada0cc0bece53f1: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:44:45.839454 6271 tablet_replica.cc:439] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "47488393290b420cbada0cc0bece53f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } } }
I20260808 02:44:45.839708 6271 tablet_replica.cc:442] T ea23c6d4eb5c4ecb9b72b31267611d65 P be7a04ccfc364a55844a3b73a54a44e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:44:45.844620 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.831145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48848 (local address 127.25.137.193:42821)
0808 02:44:45.837450 (+ 6305us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.837456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.837476 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:44:45.837573 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.837579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.837686 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.838492 (+ 806us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:45.838900 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.840172 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.842279 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.842794 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.842893 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.843399 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:45.844000 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:45.844109 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.844235 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.844319 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6165,"thread_start_us":6039,"threads_started":1}
I20260808 02:44:45.860320 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.855004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:48697 (local address 127.25.137.194:42207)
0808 02:44:45.855172 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.855176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.855658 (+ 482us) server_negotiation.cc:407] Connection header received
0808 02:44:45.855841 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.855845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.855915 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.855987 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:45.856454 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.857303 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.858629 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.859064 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.859191 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.859594 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:45.859709 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:45.859792 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.859917 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.859999 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:44:45.860388 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.854909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:42207 (local address 127.25.137.193:48697)
0808 02:44:45.855558 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:45.855572 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:45.855705 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:45.856015 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:45.856019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:45.856031 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:44:45.856313 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.856322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.857427 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.857431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.858492 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.858503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.859237 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.859244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.859381 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.859433 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:44:45.859800 (+ 367us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:44:45.859812 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 02:44:45.859882 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:44:45.860021 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":394,"threads_started":1}
I20260808 02:44:45.870961 6150 raft_consensus.cc:1261] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Refusing update from remote peer 8afcd23c51b249c3912614f4d62f07d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:45.871341 5931 raft_consensus.cc:1261] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Refusing update from remote peer 8afcd23c51b249c3912614f4d62f07d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:45.871622 6314 consensus_queue.cc:1035] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:45.872159 6270 consensus_queue.cc:1035] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:45.876523 6342 mvcc.cc:205] Tried to move back new op lower bound from 7316099423721676800 to 7316099422776471552. Current Snapshot: MvccSnapshot[applied={T|T < 7316099423721676800}]
I20260808 02:44:45.877094 6344 mvcc.cc:205] Tried to move back new op lower bound from 7316099423721676800 to 7316099422776471552. Current Snapshot: MvccSnapshot[applied={T|T < 7316099423721676800}]
I20260808 02:44:45.917659 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.910548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:59849 (local address 127.25.137.193:42821)
0808 02:44:45.910719 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:45.910725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:45.911499 (+ 774us) server_negotiation.cc:407] Connection header received
0808 02:44:45.911752 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:45.911757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:45.911847 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:45.911933 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:45.912669 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.913500 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.915648 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.916240 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.916385 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.916873 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:45.916974 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:45.917092 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:45.917260 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 02:44:45.917348 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:44:45.917796 6374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:45.910605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42821 (local address 127.25.137.194:59849)
0808 02:44:45.911373 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:45.911393 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:45.911563 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:45.912112 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:45.912118 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:45.912131 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:44:45.912498 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.912511 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.913623 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.913628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.915511 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:45.915528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:45.916520 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:45.916526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:45.916647 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:45.916701 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:44:45.917146 (+ 445us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:44:45.917159 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 02:44:45.917232 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 02:44:45.917369 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":394,"threads_started":1}
I20260808 02:44:45.966524 5453 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:45.966849 5453 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:45.970916 5453 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5: Sending DeleteTablet for 3 replicas of tablet 11b3bda6a79c443b9d025ef6c0d6325a
I20260808 02:44:45.972374 5677 tablet_service.cc:1514] Processing DeleteTablet for tablet 11b3bda6a79c443b9d025ef6c0d6325a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:45 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:48832
I20260808 02:44:45.972777 6109 tablet_service.cc:1514] Processing DeleteTablet for tablet 11b3bda6a79c443b9d025ef6c0d6325a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:45 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:57032
I20260808 02:44:45.973424 5894 tablet_service.cc:1514] Processing DeleteTablet for tablet 11b3bda6a79c443b9d025ef6c0d6325a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:45 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:48716
I20260808 02:44:45.974308 6379 tablet_replica.cc:330] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4: stopping tablet replica
I20260808 02:44:45.974566 6379 raft_consensus.cc:2229] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:45.974912 6379 raft_consensus.cc:2258] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:45.976114 6379 ts_tablet_manager.cc:1905] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:45.981662 6382 tablet_replica.cc:330] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4: stopping tablet replica
I20260808 02:44:45.981884 6382 raft_consensus.cc:2229] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:44:45.982524 6382 raft_consensus.cc:2258] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:45.982863 6379 ts_tablet_manager.cc:1918] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:44:45.983060 6379 log.cc:1196] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/11b3bda6a79c443b9d025ef6c0d6325a
I20260808 02:44:45.983605 6379 ts_tablet_manager.cc:1939] T 11b3bda6a79c443b9d025ef6c0d6325a P be7a04ccfc364a55844a3b73a54a44e4: Deleting consensus metadata
I20260808 02:44:45.984812 5429 catalog_manager.cc:4841] TS be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633): tablet 11b3bda6a79c443b9d025ef6c0d6325a (table test-table [id=4452d9d295c042b4a6867d39d3fc88ac]) successfully deleted
I20260808 02:44:45.984916 6382 ts_tablet_manager.cc:1905] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:46.013480 6380 tablet_replica.cc:330] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1: stopping tablet replica
I20260808 02:44:46.013736 6380 raft_consensus.cc:2229] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:46.014061 6380 raft_consensus.cc:2258] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:46.015206 6382 ts_tablet_manager.cc:1918] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:44:46.015249 6380 ts_tablet_manager.cc:1905] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:46.015389 6382 log.cc:1196] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/11b3bda6a79c443b9d025ef6c0d6325a
I20260808 02:44:46.017724 6382 ts_tablet_manager.cc:1939] T 11b3bda6a79c443b9d025ef6c0d6325a P 8afcd23c51b249c3912614f4d62f07d4: Deleting consensus metadata
I20260808 02:44:46.018990 5429 catalog_manager.cc:4841] TS 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821): tablet 11b3bda6a79c443b9d025ef6c0d6325a (table test-table [id=4452d9d295c042b4a6867d39d3fc88ac]) successfully deleted
I20260808 02:44:46.023299 6380 ts_tablet_manager.cc:1918] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:44:46.023452 6380 log.cc:1196] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/11b3bda6a79c443b9d025ef6c0d6325a
I20260808 02:44:46.024046 6380 ts_tablet_manager.cc:1939] T 11b3bda6a79c443b9d025ef6c0d6325a P 47488393290b420cbada0cc0bece53f1: Deleting consensus metadata
I20260808 02:44:46.025261 5431 catalog_manager.cc:4841] TS 47488393290b420cbada0cc0bece53f1 (127.25.137.194:42207): tablet 11b3bda6a79c443b9d025ef6c0d6325a (table test-table [id=4452d9d295c042b4a6867d39d3fc88ac]) successfully deleted
Aug 08 02:44:46 dist-test-slave-k5rr krb5kdc[5379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:44:46 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:46 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:44:46.084246 6327 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:46.071035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49094 (local address 127.25.137.254:36945)
0808 02:44:46.071227 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:46.071233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:46.071781 (+ 548us) server_negotiation.cc:407] Connection header received
0808 02:44:46.071997 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:46.072003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:46.072110 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:46.072225 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:46.074008 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.075182 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.077145 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.077832 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.077973 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.083980 (+ 6007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49094: BlockingRecv error: recv got EOF from 127.0.0.1:49094 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 02:44:46 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM
I20260808 02:44:46.106056 6327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:46.087857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.25.137.254:36945)
0808 02:44:46.088035 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:46.088040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:46.088546 (+ 506us) server_negotiation.cc:407] Connection header received
0808 02:44:46.088714 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:46.088719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:46.088834 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:46.088939 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:46.090476 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.091645 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.093530 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.094181 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.094315 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.099283 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:46.099291 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:46.099296 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:46.099360 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:46.102966 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:46.103770 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:46.103776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:46.103780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:46.103938 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:46.104378 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:46.104383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:46.104387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:46.104773 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:46.105060 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:46.105408 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 02:44:46.105671 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:44:46.110176 5453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:44:46.110927 5453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:44:46.118870 5677 tablet_service.cc:1467] Processing CreateTablet for tablet e73c381cf0ad44c28084ef90d772c271 (DEFAULT_TABLE table=test-table [id=ef1eb86c44ce43ff8fff2cbfe53abdd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:46.119382 5677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e73c381cf0ad44c28084ef90d772c271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:46.119277 5894 tablet_service.cc:1467] Processing CreateTablet for tablet e73c381cf0ad44c28084ef90d772c271 (DEFAULT_TABLE table=test-table [id=ef1eb86c44ce43ff8fff2cbfe53abdd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:46.119767 5894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e73c381cf0ad44c28084ef90d772c271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:46.126361 6109 tablet_service.cc:1467] Processing CreateTablet for tablet e73c381cf0ad44c28084ef90d772c271 (DEFAULT_TABLE table=test-table [id=ef1eb86c44ce43ff8fff2cbfe53abdd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:46.126868 6109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e73c381cf0ad44c28084ef90d772c271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:46.127195 6263 tablet_bootstrap.cc:492] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1: Bootstrap starting.
I20260808 02:44:46.128994 6263 tablet_bootstrap.cc:654] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:46.136478 6260 tablet_bootstrap.cc:492] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4: Bootstrap starting.
I20260808 02:44:46.138224 6260 tablet_bootstrap.cc:654] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:46.144295 6258 tablet_bootstrap.cc:492] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4: Bootstrap starting.
I20260808 02:44:46.146148 6258 tablet_bootstrap.cc:654] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:46.152164 6260 tablet_bootstrap.cc:492] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4: No bootstrap required, opened a new log
I20260808 02:44:46.152403 6260 ts_tablet_manager.cc:1397] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.006s
I20260808 02:44:46.153335 6260 raft_consensus.cc:348] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.153777 6260 raft_consensus.cc:374] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:46.153926 6260 raft_consensus.cc:729] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be7a04ccfc364a55844a3b73a54a44e4, State: Initialized, Role: FOLLOWER
I20260808 02:44:46.154251 6260 consensus_queue.cc:260] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.154973 6260 ts_tablet_manager.cc:1428] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:44:46.158643 6263 tablet_bootstrap.cc:492] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1: No bootstrap required, opened a new log
I20260808 02:44:46.158828 6263 ts_tablet_manager.cc:1397] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1: Time spent bootstrapping tablet: real 0.032s user 0.023s sys 0.004s
I20260808 02:44:46.159725 6263 raft_consensus.cc:348] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.160072 6263 raft_consensus.cc:374] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:46.160204 6263 raft_consensus.cc:729] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47488393290b420cbada0cc0bece53f1, State: Initialized, Role: FOLLOWER
I20260808 02:44:46.160481 6263 consensus_queue.cc:260] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.161105 6263 ts_tablet_manager.cc:1428] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 02:44:46.163024 6271 raft_consensus.cc:482] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:46.163218 6271 raft_consensus.cc:504] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.165165 6271 leader_election.cc:288] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821), 47488393290b420cbada0cc0bece53f1 (127.25.137.194:42207)
I20260808 02:44:46.165534 6258 tablet_bootstrap.cc:492] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4: No bootstrap required, opened a new log
I20260808 02:44:46.165720 6258 ts_tablet_manager.cc:1397] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.006s
I20260808 02:44:46.166531 6258 raft_consensus.cc:348] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.166815 6258 raft_consensus.cc:374] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:46.166921 6258 raft_consensus.cc:729] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afcd23c51b249c3912614f4d62f07d4, State: Initialized, Role: FOLLOWER
I20260808 02:44:46.167186 6258 consensus_queue.cc:260] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.167776 6258 ts_tablet_manager.cc:1428] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:44:46.188094 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:46.165699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:38219 (local address 127.25.137.193:42821)
0808 02:44:46.166079 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:46.166086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:46.179281 (+ 13195us) server_negotiation.cc:407] Connection header received
0808 02:44:46.179527 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:46.179534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:46.179629 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:46.179721 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:46.180395 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.181588 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.185543 (+ 3955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.186881 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.187003 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.187077 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:46.187605 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 02:44:46.187772 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228}
I20260808 02:44:46.188935 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:46.165325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42821 (local address 127.25.137.195:38219)
0808 02:44:46.179148 (+ 13823us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:46.179171 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:46.179344 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:46.179929 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:46.179936 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:46.179970 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:46.180204 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:46.180218 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.181803 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.181808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:46.185278 (+ 3470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:46.185294 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.187233 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.187239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:46.187417 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.187451 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:44:46.188364 (+ 913us) client_negotiation.cc:240] Negotiation successful
0808 02:44:46.188510 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13672,"thread_start_us":15839,"threads_started":1}
I20260808 02:44:46.189743 5702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73c381cf0ad44c28084ef90d772c271" candidate_uuid: "be7a04ccfc364a55844a3b73a54a44e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcd23c51b249c3912614f4d62f07d4" is_pre_election: true
I20260808 02:44:46.190043 5702 raft_consensus.cc:2454] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be7a04ccfc364a55844a3b73a54a44e4 in term 0.
I20260808 02:44:46.190785 6406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:46.165689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:42207 (local address 127.25.137.195:56149)
0808 02:44:46.181465 (+ 15776us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:46.181480 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:46.181595 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:46.182079 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:46.182084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:46.182112 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:44:46.182475 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:46.182488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.184002 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.184007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:46.186187 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:46.186203 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.188120 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.188126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:46.188511 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.188548 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:44:46.188640 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 02:44:46.188768 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15678,"thread_start_us":13134,"threads_started":1}
I20260808 02:44:46.191210 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:46.165863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:56149 (local address 127.25.137.194:42207)
0808 02:44:46.166229 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:46.166236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:46.181581 (+ 15345us) server_negotiation.cc:407] Connection header received
0808 02:44:46.181705 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:46.181710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:46.181800 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:46.181887 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:44:46.182681 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.183820 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.186431 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.187715 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.187903 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.187973 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:46.189233 (+ 1260us) server_negotiation.cc:299] Negotiation successful
0808 02:44:46.189399 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248}
I20260808 02:44:46.190735 6029 leader_election.cc:302] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8afcd23c51b249c3912614f4d62f07d4, be7a04ccfc364a55844a3b73a54a44e4; no voters:
I20260808 02:44:46.191984 6271 raft_consensus.cc:2781] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:44:46.192095 6271 raft_consensus.cc:482] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:44:46.192179 6271 raft_consensus.cc:3037] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:46.192350 5931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73c381cf0ad44c28084ef90d772c271" candidate_uuid: "be7a04ccfc364a55844a3b73a54a44e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47488393290b420cbada0cc0bece53f1" is_pre_election: true
I20260808 02:44:46.192600 5931 raft_consensus.cc:2454] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be7a04ccfc364a55844a3b73a54a44e4 in term 0.
I20260808 02:44:46.195420 6271 raft_consensus.cc:504] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.197324 6271 leader_election.cc:288] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [CANDIDATE]: Term 1 election: Requested vote from peers 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821), 47488393290b420cbada0cc0bece53f1 (127.25.137.194:42207)
I20260808 02:44:46.197122 5931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73c381cf0ad44c28084ef90d772c271" candidate_uuid: "be7a04ccfc364a55844a3b73a54a44e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47488393290b420cbada0cc0bece53f1"
I20260808 02:44:46.197682 5931 raft_consensus.cc:3037] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:46.200981 5931 raft_consensus.cc:2454] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be7a04ccfc364a55844a3b73a54a44e4 in term 1.
I20260808 02:44:46.201306 5702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73c381cf0ad44c28084ef90d772c271" candidate_uuid: "be7a04ccfc364a55844a3b73a54a44e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcd23c51b249c3912614f4d62f07d4"
I20260808 02:44:46.201500 6031 leader_election.cc:302] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47488393290b420cbada0cc0bece53f1, be7a04ccfc364a55844a3b73a54a44e4; no voters:
I20260808 02:44:46.201547 5702 raft_consensus.cc:3037] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:46.201828 6271 raft_consensus.cc:2781] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:46.204613 5702 raft_consensus.cc:2454] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be7a04ccfc364a55844a3b73a54a44e4 in term 1.
I20260808 02:44:46.205560 6271 raft_consensus.cc:686] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 1 LEADER]: Becoming Leader. State: Replica: be7a04ccfc364a55844a3b73a54a44e4, State: Running, Role: LEADER
I20260808 02:44:46.206023 6271 consensus_queue.cc:237] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.210243 5454 catalog_manager.cc:5495] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195). New cstate: current_term: 1 leader_uuid: "be7a04ccfc364a55844a3b73a54a44e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } health_report { overall_health: HEALTHY } } }
I20260808 02:44:46.240513 6413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:46.232498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57044 (local address 127.25.137.195:46633)
0808 02:44:46.233436 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:46.233442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:46.233461 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:44:46.233565 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:46.233570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:46.233676 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:46.233874 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:46.234299 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.235687 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.237689 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.238379 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.238513 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.239095 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:46.239849 (+ 754us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:46.240005 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:46.240061 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 02:44:46.240152 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":698,"threads_started":1}
I20260808 02:44:46.244683 5702 raft_consensus.cc:1261] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Refusing update from remote peer be7a04ccfc364a55844a3b73a54a44e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:46.244745 5931 raft_consensus.cc:1261] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Refusing update from remote peer be7a04ccfc364a55844a3b73a54a44e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:46.245604 6409 consensus_queue.cc:1035] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:46.246064 6271 consensus_queue.cc:1035] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:46.282732 5454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:46.283084 5454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:46.286705 5454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5: Sending DeleteTablet for 3 replicas of tablet e73c381cf0ad44c28084ef90d772c271
I20260808 02:44:46.287691 5677 tablet_service.cc:1514] Processing DeleteTablet for tablet e73c381cf0ad44c28084ef90d772c271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:46 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:48832
I20260808 02:44:46.288034 6382 tablet_replica.cc:330] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4: stopping tablet replica
I20260808 02:44:46.288261 6382 raft_consensus.cc:2229] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:46.288519 6382 raft_consensus.cc:2258] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:46.288774 5894 tablet_service.cc:1514] Processing DeleteTablet for tablet e73c381cf0ad44c28084ef90d772c271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:46 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:48716
I20260808 02:44:46.289130 6380 tablet_replica.cc:330] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1: stopping tablet replica
I20260808 02:44:46.289361 6380 raft_consensus.cc:2229] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:46.289598 6380 raft_consensus.cc:2258] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:46.290066 6382 ts_tablet_manager.cc:1905] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:46.291371 6380 ts_tablet_manager.cc:1905] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:46.291916 6109 tablet_service.cc:1514] Processing DeleteTablet for tablet e73c381cf0ad44c28084ef90d772c271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:46 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:57032
I20260808 02:44:46.292265 6379 tablet_replica.cc:330] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4: stopping tablet replica
I20260808 02:44:46.292490 6379 raft_consensus.cc:2229] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:44:46.292968 6379 raft_consensus.cc:2258] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:46.294174 6379 ts_tablet_manager.cc:1905] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:46.294581 5454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:44:46.295410 5454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:44:46.296722 6382 ts_tablet_manager.cc:1918] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:46.296927 6382 log.cc:1196] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/e73c381cf0ad44c28084ef90d772c271
I20260808 02:44:46.297397 6382 ts_tablet_manager.cc:1939] T e73c381cf0ad44c28084ef90d772c271 P 8afcd23c51b249c3912614f4d62f07d4: Deleting consensus metadata
I20260808 02:44:46.299142 5429 catalog_manager.cc:4841] TS 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821): tablet e73c381cf0ad44c28084ef90d772c271 (table test-table [id=ef1eb86c44ce43ff8fff2cbfe53abdd8]) successfully deleted
I20260808 02:44:46.300616 6380 ts_tablet_manager.cc:1918] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:46.300798 6380 log.cc:1196] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/e73c381cf0ad44c28084ef90d772c271
I20260808 02:44:46.301241 6380 ts_tablet_manager.cc:1939] T e73c381cf0ad44c28084ef90d772c271 P 47488393290b420cbada0cc0bece53f1: Deleting consensus metadata
I20260808 02:44:46.302342 5431 catalog_manager.cc:4841] TS 47488393290b420cbada0cc0bece53f1 (127.25.137.194:42207): tablet e73c381cf0ad44c28084ef90d772c271 (table test-table [id=ef1eb86c44ce43ff8fff2cbfe53abdd8]) successfully deleted
I20260808 02:44:46.307291 5894 tablet_service.cc:1467] Processing CreateTablet for tablet d81ed122b8e94a1f84a3df89419488f5 (DEFAULT_TABLE table=test-table [id=1b051d4e93de41b0b1e9bba0efe58ef9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:46.307741 5894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d81ed122b8e94a1f84a3df89419488f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:46.313356 6379 ts_tablet_manager.cc:1918] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:44:46.313561 6379 log.cc:1196] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/e73c381cf0ad44c28084ef90d772c271
I20260808 02:44:46.314067 6379 ts_tablet_manager.cc:1939] T e73c381cf0ad44c28084ef90d772c271 P be7a04ccfc364a55844a3b73a54a44e4: Deleting consensus metadata
I20260808 02:44:46.313997 6109 tablet_service.cc:1467] Processing CreateTablet for tablet d81ed122b8e94a1f84a3df89419488f5 (DEFAULT_TABLE table=test-table [id=1b051d4e93de41b0b1e9bba0efe58ef9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:46.315099 6109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d81ed122b8e94a1f84a3df89419488f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:46.317581 6263 tablet_bootstrap.cc:492] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1: Bootstrap starting.
I20260808 02:44:46.317832 5429 catalog_manager.cc:4841] TS be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633): tablet e73c381cf0ad44c28084ef90d772c271 (table test-table [id=ef1eb86c44ce43ff8fff2cbfe53abdd8]) successfully deleted
I20260808 02:44:46.318120 5677 tablet_service.cc:1467] Processing CreateTablet for tablet d81ed122b8e94a1f84a3df89419488f5 (DEFAULT_TABLE table=test-table [id=1b051d4e93de41b0b1e9bba0efe58ef9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:44:46.318645 5677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d81ed122b8e94a1f84a3df89419488f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:46.319278 6263 tablet_bootstrap.cc:654] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:46.326184 6258 tablet_bootstrap.cc:492] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4: Bootstrap starting.
I20260808 02:44:46.328049 6258 tablet_bootstrap.cc:654] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:46.331370 6260 tablet_bootstrap.cc:492] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4: Bootstrap starting.
I20260808 02:44:46.333398 6260 tablet_bootstrap.cc:654] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:46.333730 6263 tablet_bootstrap.cc:492] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1: No bootstrap required, opened a new log
I20260808 02:44:46.333953 6263 ts_tablet_manager.cc:1397] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.001s
I20260808 02:44:46.334913 6263 raft_consensus.cc:348] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.335233 6263 raft_consensus.cc:374] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:46.335327 6263 raft_consensus.cc:729] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47488393290b420cbada0cc0bece53f1, State: Initialized, Role: FOLLOWER
I20260808 02:44:46.335628 6263 consensus_queue.cc:260] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.336269 6263 ts_tablet_manager.cc:1428] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 02:44:46.341535 6258 tablet_bootstrap.cc:492] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4: No bootstrap required, opened a new log
I20260808 02:44:46.341737 6258 ts_tablet_manager.cc:1397] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.005s
I20260808 02:44:46.342690 6260 tablet_bootstrap.cc:492] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4: No bootstrap required, opened a new log
I20260808 02:44:46.342713 6258 raft_consensus.cc:348] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.342902 6260 ts_tablet_manager.cc:1397] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260808 02:44:46.343000 6258 raft_consensus.cc:374] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:46.343096 6258 raft_consensus.cc:729] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afcd23c51b249c3912614f4d62f07d4, State: Initialized, Role: FOLLOWER
I20260808 02:44:46.343432 6258 consensus_queue.cc:260] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.343967 6260 raft_consensus.cc:348] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.344257 6260 raft_consensus.cc:374] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:46.344354 6260 raft_consensus.cc:729] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be7a04ccfc364a55844a3b73a54a44e4, State: Initialized, Role: FOLLOWER
I20260808 02:44:46.344619 6260 consensus_queue.cc:260] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.345343 6260 ts_tablet_manager.cc:1428] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:44:46.346700 6258 ts_tablet_manager.cc:1428] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260808 02:44:46.574373 6334 raft_consensus.cc:482] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:44:46.574590 6334 raft_consensus.cc:504] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.575192 6334 leader_election.cc:288] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821), be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633)
I20260808 02:44:46.575915 6150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d81ed122b8e94a1f84a3df89419488f5" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be7a04ccfc364a55844a3b73a54a44e4" is_pre_election: true
I20260808 02:44:46.575906 5702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d81ed122b8e94a1f84a3df89419488f5" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcd23c51b249c3912614f4d62f07d4" is_pre_election: true
I20260808 02:44:46.576186 6150 raft_consensus.cc:2454] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47488393290b420cbada0cc0bece53f1 in term 0.
I20260808 02:44:46.576184 5702 raft_consensus.cc:2454] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47488393290b420cbada0cc0bece53f1 in term 0.
I20260808 02:44:46.576644 5817 leader_election.cc:302] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47488393290b420cbada0cc0bece53f1, 8afcd23c51b249c3912614f4d62f07d4; no voters:
I20260808 02:44:46.576930 6334 raft_consensus.cc:2781] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:44:46.577060 6334 raft_consensus.cc:482] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:44:46.577175 6334 raft_consensus.cc:3037] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:46.579176 6334 raft_consensus.cc:504] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.579711 6334 leader_election.cc:288] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 election: Requested vote from peers 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821), be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633)
I20260808 02:44:46.580214 5702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d81ed122b8e94a1f84a3df89419488f5" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcd23c51b249c3912614f4d62f07d4"
I20260808 02:44:46.580415 5702 raft_consensus.cc:3037] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:46.580368 6150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d81ed122b8e94a1f84a3df89419488f5" candidate_uuid: "47488393290b420cbada0cc0bece53f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be7a04ccfc364a55844a3b73a54a44e4"
I20260808 02:44:46.580590 6150 raft_consensus.cc:3037] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:46.582738 5702 raft_consensus.cc:2454] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47488393290b420cbada0cc0bece53f1 in term 1.
I20260808 02:44:46.583145 5817 leader_election.cc:302] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47488393290b420cbada0cc0bece53f1, 8afcd23c51b249c3912614f4d62f07d4; no voters:
I20260808 02:44:46.583395 6334 raft_consensus.cc:2781] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:46.583796 6150 raft_consensus.cc:2454] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47488393290b420cbada0cc0bece53f1 in term 1.
I20260808 02:44:46.583811 6334 raft_consensus.cc:686] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 1 LEADER]: Becoming Leader. State: Replica: 47488393290b420cbada0cc0bece53f1, State: Running, Role: LEADER
I20260808 02:44:46.584100 6334 consensus_queue.cc:237] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } }
I20260808 02:44:46.586810 5453 catalog_manager.cc:5495] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47488393290b420cbada0cc0bece53f1 (127.25.137.194). New cstate: current_term: 1 leader_uuid: "47488393290b420cbada0cc0bece53f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47488393290b420cbada0cc0bece53f1" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 } health_report { overall_health: UNKNOWN } } }
I20260808 02:44:46.642408 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:46.636358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.25.137.194:42207)
0808 02:44:46.636694 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:46.636701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:46.636718 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:44:46.636783 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:46.636788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:46.636873 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:46.636966 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:46.637498 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.638638 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.640554 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.641044 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.641160 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.641658 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:46.641775 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:46.641883 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:46.642019 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 02:44:46.642102 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260808 02:44:46.669582 6434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:46.650380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:42207 (local address 127.25.137.194:34917)
0808 02:44:46.663902 (+ 13522us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:46.663920 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:46.664090 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:46.664506 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:46.664510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:46.664520 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:44:46.664731 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:46.664739 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.666134 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.666138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:46.667346 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:46.667362 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.668413 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.668418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:46.668544 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.668584 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:44:46.669053 (+ 469us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:44:46.669066 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 02:44:46.669132 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 02:44:46.669238 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13368,"thread_start_us":13674,"threads_started":1}
I20260808 02:44:46.669588 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:46.650606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:34917 (local address 127.25.137.194:42207)
0808 02:44:46.650823 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:46.650830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:46.664027 (+ 13197us) server_negotiation.cc:407] Connection header received
0808 02:44:46.664286 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:46.664291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:46.664383 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:46.664490 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:44:46.664879 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.665961 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.667500 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.668215 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.668357 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.668761 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:44:46.668898 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:44:46.668993 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:46.669186 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 02:44:46.669277 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:44:46.678045 6150 raft_consensus.cc:1261] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Refusing update from remote peer 47488393290b420cbada0cc0bece53f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:46.678131 5702 raft_consensus.cc:1261] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Refusing update from remote peer 47488393290b420cbada0cc0bece53f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:44:46.678689 6429 consensus_queue.cc:1035] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8afcd23c51b249c3912614f4d62f07d4" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:46.679016 6334 consensus_queue.cc:1035] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be7a04ccfc364a55844a3b73a54a44e4" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:44:46.739116 5457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:46.739419 5457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:44:46.742744 5457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ac89efa23664878b1546a04e24c67d5: Sending DeleteTablet for 3 replicas of tablet d81ed122b8e94a1f84a3df89419488f5
I20260808 02:44:46.743834 5892 tablet_service.cc:1514] Processing DeleteTablet for tablet d81ed122b8e94a1f84a3df89419488f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:46 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:48716
I20260808 02:44:46.743882 5677 tablet_service.cc:1514] Processing DeleteTablet for tablet d81ed122b8e94a1f84a3df89419488f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:46 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:48832
I20260808 02:44:46.743884 6109 tablet_service.cc:1514] Processing DeleteTablet for tablet d81ed122b8e94a1f84a3df89419488f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:44:46 UTC) from {username='oryx', principal='oryx/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:57032
I20260808 02:44:46.744175 6380 tablet_replica.cc:330] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1: stopping tablet replica
I20260808 02:44:46.744194 6379 tablet_replica.cc:330] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4: stopping tablet replica
I20260808 02:44:46.744372 6380 raft_consensus.cc:2229] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:44:46.744374 6379 raft_consensus.cc:2229] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:46.744583 6379 raft_consensus.cc:2258] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:46.744704 6380 raft_consensus.cc:2258] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:46.745000 6382 tablet_replica.cc:330] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4: stopping tablet replica
I20260808 02:44:46.745209 6382 raft_consensus.cc:2229] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:44:46.745429 6382 raft_consensus.cc:2258] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:44:46.745586 6379 ts_tablet_manager.cc:1905] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:46.746416 6380 ts_tablet_manager.cc:1905] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:46.750361 6382 ts_tablet_manager.cc:1905] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:44:46.751080 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 5571
I20260808 02:44:46.751941 6380 ts_tablet_manager.cc:1918] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:44:46.752127 6380 log.cc:1196] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/d81ed122b8e94a1f84a3df89419488f5
I20260808 02:44:46.752605 6380 ts_tablet_manager.cc:1939] T d81ed122b8e94a1f84a3df89419488f5 P 47488393290b420cbada0cc0bece53f1: Deleting consensus metadata
I20260808 02:44:46.753855 5431 catalog_manager.cc:4841] TS 47488393290b420cbada0cc0bece53f1 (127.25.137.194:42207): tablet d81ed122b8e94a1f84a3df89419488f5 (table test-table [id=1b051d4e93de41b0b1e9bba0efe58ef9]) successfully deleted
I20260808 02:44:46.755607 6379 ts_tablet_manager.cc:1918] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:44:46.755752 6379 log.cc:1196] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/d81ed122b8e94a1f84a3df89419488f5
I20260808 02:44:46.756140 6379 ts_tablet_manager.cc:1939] T d81ed122b8e94a1f84a3df89419488f5 P be7a04ccfc364a55844a3b73a54a44e4: Deleting consensus metadata
I20260808 02:44:46.756947 6382 ts_tablet_manager.cc:1918] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:44:46.757141 6382 log.cc:1196] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/d81ed122b8e94a1f84a3df89419488f5
I20260808 02:44:46.757263 5429 catalog_manager.cc:4841] TS be7a04ccfc364a55844a3b73a54a44e4 (127.25.137.195:46633): tablet d81ed122b8e94a1f84a3df89419488f5 (table test-table [id=1b051d4e93de41b0b1e9bba0efe58ef9]) successfully deleted
I20260808 02:44:46.757551 6382 ts_tablet_manager.cc:1939] T d81ed122b8e94a1f84a3df89419488f5 P 8afcd23c51b249c3912614f4d62f07d4: Deleting consensus metadata
I20260808 02:44:46.758560 5429 catalog_manager.cc:4841] TS 8afcd23c51b249c3912614f4d62f07d4 (127.25.137.193:42821): tablet d81ed122b8e94a1f84a3df89419488f5 (table test-table [id=1b051d4e93de41b0b1e9bba0efe58ef9]) successfully deleted
Aug 08 02:44:46 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.137.193@KRBTEST.COM
I20260808 02:44:46.768630 6442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:46.752346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48864 (local address 127.25.137.193:42821)
0808 02:44:46.754742 (+ 2396us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:46.754748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:46.754768 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:44:46.754816 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:46.754820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:46.754886 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:46.754988 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:46.756302 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.757069 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.758779 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.759181 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.759313 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.763073 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:46.763087 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:46.763104 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:46.763133 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:46.765890 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:46.766555 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:46.766563 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:46.766565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:46.766697 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:46.767122 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:46.767129 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:46.767132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:46.767560 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:46.767796 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:46.768067 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 02:44:46.768262 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2300,"thread_start_us":2178,"threads_started":1}
I20260808 02:44:46.769294 5777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:46.892485 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 5571
I20260808 02:44:46.923076 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 5794
Aug 08 02:44:46 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.137.194@KRBTEST.COM
I20260808 02:44:46.937610 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:46.923874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48760 (local address 127.25.137.194:42207)
0808 02:44:46.924044 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:46.924048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:46.924146 (+ 98us) server_negotiation.cc:407] Connection header received
0808 02:44:46.924304 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:46.924327 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:46.924393 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:46.924475 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:46.925728 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.926407 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.927741 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:46.928124 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:46.928208 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:46.932293 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:46.932323 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:46.932340 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:46.932377 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:46.935032 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:46.935672 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:46.935678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:46.935681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:46.935805 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:46.936087 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:46.936090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:46.936092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:46.936437 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:46.936698 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:46.937120 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 02:44:46.937295 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:44:46.938357 5988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:47.090384 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 5794
I20260808 02:44:47.124630 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 6006
Aug 08 02:44:47 dist-test-slave-k5rr krb5kdc[5379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.137.195@KRBTEST.COM
I20260808 02:44:47.139956 6447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:47.125487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57050 (local address 127.25.137.195:46633)
0808 02:44:47.125971 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:47.125975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:47.125992 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:44:47.126048 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:47.126051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:47.126122 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:47.126208 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:47.127467 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:47.128188 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:47.129472 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:47.129891 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:47.129993 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:47.134453 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:47.134471 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:47.134488 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:47.134525 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:47.137293 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:47.137993 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:47.138001 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:47.138004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:47.138155 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:47.138515 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:47.138518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:47.138520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:47.138875 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:47.139112 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:47.139372 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 02:44:47.139553 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":261,"threads_started":1}
I20260808 02:44:47.140618 6208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:47.265918 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 6006
I20260808 02:44:47.292230 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 5406
I20260808 02:44:47.293488 5526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:47.410624 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 5406
2026-08-08T02:44:47Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6042 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:44:47 dist-test-slave-k5rr krb5kdc[6458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:44:47 dist-test-slave-k5rr krb5kdc[6458](info): set up 2 sockets
Aug 08 02:44:47 dist-test-slave-k5rr krb5kdc[6458](info): commencing operation
krb5kdc: starting...
W20260808 02:44:48.635041 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:44:48 dist-test-slave-k5rr krb5kdc[6458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:44:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:44:48.890837 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:40869
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:46719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:40869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:49.219331 6481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:49.219604 6481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:49.219739 6481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:49.227010 6481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:49.227123 6481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:49.227186 6481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:49.227264 6481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:49.227322 6481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:49.237437 6481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:46719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:40869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:40869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:49.238711 6481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:49.240058 6481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:49.251549 6493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:49.251534 6491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:49.251713 6489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:49.252625 6481 server_base.cc:1034] running on GCE node
I20260808 02:44:49.253459 6481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:49.254841 6481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:49.256042 6481 hybrid_clock.cc:648] HybridClock initialized: now 1786157089255997 us; error 35 us; skew 500 ppm
I20260808 02:44:49.256379 6481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T02:44:49Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1903 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:44:49 dist-test-slave-k5rr krb5kdc[6503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:44:49 dist-test-slave-k5rr krb5kdc[6503](info): set up 2 sockets
Aug 08 02:44:49 dist-test-slave-k5rr krb5kdc[6503](info): commencing operation
krb5kdc: starting...
W20260808 02:44:50.554934 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:44:50 dist-test-slave-k5rr krb5kdc[6503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:44:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:44:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:44:50.818691 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:41401
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:43021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:41401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:51.146715 6518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:51.147006 6518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:51.147119 6518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:51.154301 6518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:51.154404 6518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:51.154454 6518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:51.154534 6518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:51.154603 6518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:51.164316 6518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:41401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:41401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:51.165683 6518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:51.166961 6518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:51.178350 6525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:51.178408 6526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:51.178694 6528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:51.179438 6518 server_base.cc:1034] running on GCE node
I20260808 02:44:51.180202 6518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:51.181696 6518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:51.182912 6518 hybrid_clock.cc:648] HybridClock initialized: now 1786157091182852 us; error 50 us; skew 500 ppm
I20260808 02:44:51.183238 6518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:51 dist-test-slave-k5rr krb5kdc[6503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157091, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:51.190599 6518 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:44:51.192032 6518 webserver.cc:460] Webserver started at http://127.25.137.254:46345/ using document root <none> and password file <none>
I20260808 02:44:51.192699 6518 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:51.192816 6518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:51.193156 6518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:51.196059 6518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "c42d2b4c116d41f3b2263bc2c6770445"
format_stamp: "Formatted at 2026-08-08 02:44:51 on dist-test-slave-k5rr"
server_key: "b27d2497e6e9e5c451f013d950448f20"
server_key_iv: "6734483920f6851318f27bd3feb0fddd"
server_key_version: "encryptionkey@0"
I20260808 02:44:51.196820 6518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "c42d2b4c116d41f3b2263bc2c6770445"
format_stamp: "Formatted at 2026-08-08 02:44:51 on dist-test-slave-k5rr"
server_key: "b27d2497e6e9e5c451f013d950448f20"
server_key_iv: "6734483920f6851318f27bd3feb0fddd"
server_key_version: "encryptionkey@0"
I20260808 02:44:51.201896 6518 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:44:51.205329 6538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:51.206691 6518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:44:51.206859 6518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "c42d2b4c116d41f3b2263bc2c6770445"
format_stamp: "Formatted at 2026-08-08 02:44:51 on dist-test-slave-k5rr"
server_key: "b27d2497e6e9e5c451f013d950448f20"
server_key_iv: "6734483920f6851318f27bd3feb0fddd"
server_key_version: "encryptionkey@0"
I20260808 02:44:51.207068 6518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:51.220050 6518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:51.224493 6518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:51.224769 6518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:51.241941 6518 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:41401
I20260808 02:44:51.241952 6620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:41401 every 8 connection(s)
I20260808 02:44:51.243393 6518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:51.247130 6621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:44:51.253569 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 6518
I20260808 02:44:51.253789 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:44:51.254464 26151 external_mini_cluster.cc:1428] Setting key 98570ebdccc3cfee7bda39f37a6ea50a
I20260808 02:44:51.257812 6621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445: Bootstrap starting.
I20260808 02:44:51.261138 6621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445: Neither blocks nor log segments found. Creating new log.
I20260808 02:44:51.262375 6621 log.cc:824] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:51.265528 6621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445: No bootstrap required, opened a new log
Aug 08 02:44:51 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:51.274866 6621 raft_consensus.cc:348] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } }
I20260808 02:44:51.275315 6621 raft_consensus.cc:374] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:44:51.275430 6621 raft_consensus.cc:729] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c42d2b4c116d41f3b2263bc2c6770445, State: Initialized, Role: FOLLOWER
I20260808 02:44:51.276129 6621 consensus_queue.cc:260] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } }
I20260808 02:44:51.276436 6621 raft_consensus.cc:388] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:51.276598 6621 raft_consensus.cc:482] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:51.276818 6621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:44:51.278395 6624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:51.256541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43466 (local address 127.25.137.254:41401)
0808 02:44:51.257380 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:51.257392 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:51.257431 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:44:51.259203 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:51.259228 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:51.259611 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:51.259935 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:51.261880 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:51.263286 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:51.264703 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:51.265749 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:51.265941 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:51.269722 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:51.269737 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:51.269757 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:51.269807 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:51.273776 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:51.274448 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:51.274457 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:51.274466 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:51.274672 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:51.275047 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:51.275052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:51.275055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:51.275821 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:51.276100 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:51.276498 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 02:44:51.276956 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":400,"threads_started":1}
I20260808 02:44:51.280054 6621 raft_consensus.cc:504] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } }
I20260808 02:44:51.280670 6621 leader_election.cc:302] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c42d2b4c116d41f3b2263bc2c6770445; no voters:
I20260808 02:44:51.281379 6621 leader_election.cc:288] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:44:51.281514 6628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:44:51.282349 6628 raft_consensus.cc:686] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 1 LEADER]: Becoming Leader. State: Replica: c42d2b4c116d41f3b2263bc2c6770445, State: Running, Role: LEADER
I20260808 02:44:51.282893 6628 consensus_queue.cc:237] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } }
I20260808 02:44:51.283576 6621 sys_catalog.cc:564] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:51.285658 6631 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c42d2b4c116d41f3b2263bc2c6770445. Latest consensus state: current_term: 1 leader_uuid: "c42d2b4c116d41f3b2263bc2c6770445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } } }
I20260808 02:44:51.285748 6630 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c42d2b4c116d41f3b2263bc2c6770445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } } }
I20260808 02:44:51.286043 6631 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:51.289608 6630 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:51.290212 6638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:51.295004 6638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:51.303509 6638 catalog_manager.cc:1347] Generated new cluster ID: 4447203d59074cf9995e6a674101e817
I20260808 02:44:51.303634 6638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:51.319849 6638 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:44:51.321195 6638 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:51.331300 6638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445: Generated new TSK 0
I20260808 02:44:51.332106 6638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:44:51.396993 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41401
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:43021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:51.734968 6664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:51.735252 6664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:51.735399 6664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:51.742475 6664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:51.742590 6664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:51.742743 6664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:44:51.752274 6664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:51.753680 6664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:51.754987 6664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:51.767511 6672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:51.767524 6671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:51.767519 6675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:51.768729 6664 server_base.cc:1034] running on GCE node
I20260808 02:44:51.769373 6664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:51.770259 6664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:51.771450 6664 hybrid_clock.cc:648] HybridClock initialized: now 1786157091771409 us; error 31 us; skew 500 ppm
I20260808 02:44:51.771754 6664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:51 dist-test-slave-k5rr krb5kdc[6503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157091, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:51.774859 6664 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:44:51.776206 6664 webserver.cc:460] Webserver started at http://127.25.137.193:40835/ using document root <none> and password file <none>
I20260808 02:44:51.776841 6664 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:51.776958 6664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:51.777275 6664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:51.780042 6664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "5349ab6968e14936aa388879782ec9fa"
format_stamp: "Formatted at 2026-08-08 02:44:51 on dist-test-slave-k5rr"
server_key: "8e21cb34e5dd86485f422fadf19ac4d1"
server_key_iv: "3ef6d83fc515d2106de5b7359d99ae74"
server_key_version: "encryptionkey@0"
I20260808 02:44:51.780814 6664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "5349ab6968e14936aa388879782ec9fa"
format_stamp: "Formatted at 2026-08-08 02:44:51 on dist-test-slave-k5rr"
server_key: "8e21cb34e5dd86485f422fadf19ac4d1"
server_key_iv: "3ef6d83fc515d2106de5b7359d99ae74"
server_key_version: "encryptionkey@0"
I20260808 02:44:51.785817 6664 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:44:51.789062 6685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:51.790395 6664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:44:51.790535 6664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "5349ab6968e14936aa388879782ec9fa"
format_stamp: "Formatted at 2026-08-08 02:44:51 on dist-test-slave-k5rr"
server_key: "8e21cb34e5dd86485f422fadf19ac4d1"
server_key_iv: "3ef6d83fc515d2106de5b7359d99ae74"
server_key_version: "encryptionkey@0"
I20260808 02:44:51.790716 6664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:51.807786 6664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:51.811851 6664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:51.812115 6664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:51.813061 6664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:51.814358 6664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:51.814435 6664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:51.814548 6664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:51.814599 6664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:51.846199 6664 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:38765
I20260808 02:44:51.846287 6866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:38765 every 8 connection(s)
I20260808 02:44:51.847457 6664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:44:51.855227 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 6664
I20260808 02:44:51.855450 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:44:51.856088 26151 external_mini_cluster.cc:1428] Setting key a40be11ecff7ac6275680587dbb0eefb
Aug 08 02:44:51 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157091, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:51.868222 6872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:51.850214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:51053 (local address 127.25.137.254:41401)
0808 02:44:51.850673 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:51.850678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:51.851724 (+ 1046us) server_negotiation.cc:407] Connection header received
0808 02:44:51.853592 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:51.853596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:51.853666 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:51.853760 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:51.855213 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:51.856014 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:51.857777 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:51.858277 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:51.858379 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:51.862524 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:51.862529 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:51.862533 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:51.862567 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:51.865103 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:51.865857 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:51.865861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:51.865864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:51.865985 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:51.866447 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:51.866451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:51.866453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:51.866736 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:51.866883 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:51.867624 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 02:44:51.867865 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":217,"threads_started":1}
I20260808 02:44:51.869400 6877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:51.850841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:51053)
0808 02:44:51.851551 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:51.851597 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:51.853326 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:51.853973 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:51.853981 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:51.854475 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:51.855034 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:51.855051 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:51.856158 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:51.856162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:51.857638 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:51.857653 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:51.858517 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:51.858525 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:51.858800 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:51.859556 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:51.859587 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:51.862281 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:51.865245 (+ 2964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:51.865256 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:51.865274 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:51.865712 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:51.866097 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:51.866102 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:51.866106 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:51.866320 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:51.866871 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:51.866878 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:51.867225 (+ 347us) client_negotiation.cc:769] Sending connection context
0808 02:44:51.867486 (+ 261us) client_negotiation.cc:240] Negotiation successful
0808 02:44:51.868029 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":258,"threads_started":1}
I20260808 02:44:51.870976 6868 heartbeater.cc:347] Connected to a master server at 127.25.137.254:41401
I20260808 02:44:51.871263 6868 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:51.871992 6868 heartbeater.cc:511] Master 127.25.137.254:41401 requested a full tablet report, sending...
I20260808 02:44:51.874029 6563 ts_manager.cc:194] Registered new tserver with Master: 5349ab6968e14936aa388879782ec9fa (127.25.137.193:38765)
I20260808 02:44:51.875469 6563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:51053
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:44:51.914563 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41401
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:43021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:52.259972 6882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:52.260277 6882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:52.260437 6882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:52.267941 6882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:52.268101 6882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:52.268282 6882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:44:52.278354 6882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:52.279706 6882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:52.281104 6882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:52.294838 6888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:52.294839 6891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:52.294898 6893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:52.295972 6882 server_base.cc:1034] running on GCE node
I20260808 02:44:52.296629 6882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:52.297863 6882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:52.299108 6882 hybrid_clock.cc:648] HybridClock initialized: now 1786157092299038 us; error 53 us; skew 500 ppm
I20260808 02:44:52.299433 6882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:52 dist-test-slave-k5rr krb5kdc[6503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157092, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:52.302620 6882 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:44:52.304111 6882 webserver.cc:460] Webserver started at http://127.25.137.194:46765/ using document root <none> and password file <none>
I20260808 02:44:52.304782 6882 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:52.304919 6882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:52.305251 6882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:52.308176 6882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "21acbbc1e9954ca5a1ced11cc6f57d4e"
format_stamp: "Formatted at 2026-08-08 02:44:52 on dist-test-slave-k5rr"
server_key: "a2254445d279751efd970a64bd62a8e9"
server_key_iv: "e2a63080f943af5f60ff63ed671c9ba7"
server_key_version: "encryptionkey@0"
I20260808 02:44:52.309000 6882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "21acbbc1e9954ca5a1ced11cc6f57d4e"
format_stamp: "Formatted at 2026-08-08 02:44:52 on dist-test-slave-k5rr"
server_key: "a2254445d279751efd970a64bd62a8e9"
server_key_iv: "e2a63080f943af5f60ff63ed671c9ba7"
server_key_version: "encryptionkey@0"
I20260808 02:44:52.314455 6882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 02:44:52.317960 6902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:52.319300 6882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:44:52.319459 6882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "21acbbc1e9954ca5a1ced11cc6f57d4e"
format_stamp: "Formatted at 2026-08-08 02:44:52 on dist-test-slave-k5rr"
server_key: "a2254445d279751efd970a64bd62a8e9"
server_key_iv: "e2a63080f943af5f60ff63ed671c9ba7"
server_key_version: "encryptionkey@0"
I20260808 02:44:52.319655 6882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:52.344129 6882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:52.348516 6882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:52.348821 6882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:52.349953 6882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:52.351377 6882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:52.351456 6882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:52.351594 6882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:52.351651 6882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:52.383682 6882 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:36359
I20260808 02:44:52.383738 7075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:36359 every 8 connection(s)
I20260808 02:44:52.385033 6882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:44:52.391574 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 6882
I20260808 02:44:52.391814 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:44:52.392251 26151 external_mini_cluster.cc:1428] Setting key 880f6e6ff8535f34d7bd204e974882c3
Aug 08 02:44:52 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157092, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:52.406041 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:52.387600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:41933 (local address 127.25.137.254:41401)
0808 02:44:52.388032 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:52.388037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:52.389036 (+ 999us) server_negotiation.cc:407] Connection header received
0808 02:44:52.390945 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:52.390951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:52.391031 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:52.391136 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:52.392789 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:52.393687 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:52.395559 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:52.396232 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:52.396368 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:52.399787 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:52.399793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:52.399797 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:52.399834 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:52.402970 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:52.403699 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:52.403705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:52.403708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:52.403862 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:52.404264 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:52.404268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:52.404270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:52.404594 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:52.404728 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:52.405462 (+ 734us) server_negotiation.cc:299] Negotiation successful
0808 02:44:52.405682 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":215,"threads_started":1}
I20260808 02:44:52.407147 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:52.388134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:41933)
0808 02:44:52.388846 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:52.388891 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:52.390669 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:52.391392 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:52.391403 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:52.391991 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:52.392609 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:52.392630 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:52.393898 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:52.393902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:52.395418 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:52.395433 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:52.396461 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:52.396467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:52.396669 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:52.397382 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:52.397405 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:52.399548 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:52.403112 (+ 3564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:52.403119 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:52.403132 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:52.403581 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:52.403971 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:52.403975 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:52.403977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:52.404163 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:52.404723 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:52.404730 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:52.405098 (+ 368us) client_negotiation.cc:769] Sending connection context
0808 02:44:52.405350 (+ 252us) client_negotiation.cc:240] Negotiation successful
0808 02:44:52.405760 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":274,"threads_started":1}
I20260808 02:44:52.408573 7076 heartbeater.cc:347] Connected to a master server at 127.25.137.254:41401
I20260808 02:44:52.408879 7076 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:52.409757 7076 heartbeater.cc:511] Master 127.25.137.254:41401 requested a full tablet report, sending...
I20260808 02:44:52.411623 6563 ts_manager.cc:194] Registered new tserver with Master: 21acbbc1e9954ca5a1ced11cc6f57d4e (127.25.137.194:36359)
I20260808 02:44:52.412721 6563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:41933
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:44:52.452346 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41401
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:43021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:52.793313 7085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:52.793612 7085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:52.793764 7085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:52.801306 7085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:52.801434 7085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:52.801596 7085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:44:52.811621 7085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:52.812943 7085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:52.814298 7085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:52.827498 7091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:52.827603 7092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:52.827544 7098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:52.828665 7085 server_base.cc:1034] running on GCE node
I20260808 02:44:52.829439 7085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:52.830431 7085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:52.831677 7085 hybrid_clock.cc:648] HybridClock initialized: now 1786157092831612 us; error 47 us; skew 500 ppm
I20260808 02:44:52.832051 7085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:52 dist-test-slave-k5rr krb5kdc[6503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157092, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:52.835311 7085 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:44:52.836798 7085 webserver.cc:460] Webserver started at http://127.25.137.195:40291/ using document root <none> and password file <none>
I20260808 02:44:52.837539 7085 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:52.837672 7085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:52.838006 7085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:44:52.841135 7085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "8930f6ce5c9b4ebe965d90b734d2bb85"
format_stamp: "Formatted at 2026-08-08 02:44:52 on dist-test-slave-k5rr"
server_key: "4ac52da28848aa6aa02a2cbc7b61223d"
server_key_iv: "ee0350c03fde8b65cafc1eeb3d595e1a"
server_key_version: "encryptionkey@0"
I20260808 02:44:52.842002 7085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "8930f6ce5c9b4ebe965d90b734d2bb85"
format_stamp: "Formatted at 2026-08-08 02:44:52 on dist-test-slave-k5rr"
server_key: "4ac52da28848aa6aa02a2cbc7b61223d"
server_key_iv: "ee0350c03fde8b65cafc1eeb3d595e1a"
server_key_version: "encryptionkey@0"
I20260808 02:44:52.847693 7085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 02:44:52.851239 7109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:52.852614 7085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:44:52.852833 7085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "8930f6ce5c9b4ebe965d90b734d2bb85"
format_stamp: "Formatted at 2026-08-08 02:44:52 on dist-test-slave-k5rr"
server_key: "4ac52da28848aa6aa02a2cbc7b61223d"
server_key_iv: "ee0350c03fde8b65cafc1eeb3d595e1a"
server_key_version: "encryptionkey@0"
I20260808 02:44:52.853094 7085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:52.871150 7085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:52.875800 7085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:52.876116 7085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:52.877159 7085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:52.878455 6868 heartbeater.cc:503] Master 127.25.137.254:41401 was elected leader, sending a full tablet report...
I20260808 02:44:52.878604 7085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:52.878700 7085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:52.878880 7085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:52.878966 7085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:52.918948 7085 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:36895
I20260808 02:44:52.918973 7279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:36895 every 8 connection(s)
I20260808 02:44:52.920977 7085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:44:52.928471 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 7085
I20260808 02:44:52.928717 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:44:52.929389 26151 external_mini_cluster.cc:1428] Setting key 60ef0788a26280408a000696514b0817
Aug 08 02:44:52 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157092, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:44:52.947882 7285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:52.925292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:52011)
0808 02:44:52.926160 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:52.926208 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:52.928129 (+ 1921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:52.929080 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:52.929092 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:52.929830 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:52.930609 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:52.930634 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:52.932119 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:52.932124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:52.933673 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:52.933687 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:52.934860 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:52.934865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:52.935109 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:52.935776 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:52.935799 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:52.938134 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:44:52.942101 (+ 3967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:52.942112 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:52.942129 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:52.942715 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:52.943181 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:44:52.943186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:44:52.943190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:44:52.943446 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:44:52.944729 (+ 1283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:44:52.944740 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:44:52.945309 (+ 569us) client_negotiation.cc:769] Sending connection context
0808 02:44:52.945690 (+ 381us) client_negotiation.cc:240] Negotiation successful
0808 02:44:52.946308 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":330,"threads_started":1}
I20260808 02:44:52.949272 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:52.924687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:52011 (local address 127.25.137.254:41401)
0808 02:44:52.925266 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:52.925273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:52.928394 (+ 3121us) server_negotiation.cc:407] Connection header received
0808 02:44:52.928456 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:52.928460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:52.928537 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:52.928631 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:52.930780 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:52.931963 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:52.933840 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:52.934611 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:52.934788 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:52.938381 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:52.938390 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:52.938395 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:52.938440 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:52.941935 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:52.942870 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:52.942877 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:52.942881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:52.943049 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:52.943977 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:52.943983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:52.943987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:52.944455 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:52.944637 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:52.948519 (+ 3882us) server_negotiation.cc:299] Negotiation successful
0808 02:44:52.948785 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":269,"threads_started":1}
I20260808 02:44:52.950548 7280 heartbeater.cc:347] Connected to a master server at 127.25.137.254:41401
I20260808 02:44:52.950886 7280 heartbeater.cc:464] Registering TS with master...
I20260808 02:44:52.951635 7280 heartbeater.cc:511] Master 127.25.137.254:41401 requested a full tablet report, sending...
I20260808 02:44:52.953246 6563 ts_manager.cc:194] Registered new tserver with Master: 8930f6ce5c9b4ebe965d90b734d2bb85 (127.25.137.195:36895)
I20260808 02:44:52.954447 6563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:52011
I20260808 02:44:52.958511 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab.
Entry for principal oryx/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab.
WARNING: no policy specified for oryx/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab.
Entry for principal oryx/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab.
WARNING: no policy specified for oryx/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab.
Entry for principal oryx/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab.
I20260808 02:44:53.123809 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 6664
Aug 08 02:44:53 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:44:53.140978 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:53.124924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42224 (local address 127.25.137.193:38765)
0808 02:44:53.126098 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:53.126106 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:53.126132 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:44:53.126194 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:53.126201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:53.126424 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:53.126571 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:53.127974 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:53.128993 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:53.130346 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:53.131095 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:53.131222 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:53.135484 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:53.135496 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:53.135511 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:53.135545 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:53.138423 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:53.139051 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:53.139058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:53.139060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:53.139183 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:53.139478 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:53.139481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:53.139483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:53.139845 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:53.140075 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:53.140365 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 02:44:53.140574 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1021,"thread_start_us":862,"threads_started":1}
I20260808 02:44:53.142499 6862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:53.225353 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 6664
I20260808 02:44:53.249832 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 6882
Aug 08 02:44:53 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:44:53.266088 7300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:53.250816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47584 (local address 127.25.137.194:36359)
0808 02:44:53.251457 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:53.251467 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:53.251512 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:44:53.251587 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:53.251596 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:53.251868 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:53.252017 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:53.253350 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:53.254355 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:53.255582 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:53.256115 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:53.256258 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:53.260515 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:53.260526 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:53.260539 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:53.260579 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:53.263348 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:53.264002 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:53.264009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:53.264011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:53.264130 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:53.264489 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:53.264493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:53.264495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:53.264847 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:53.265163 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:53.265477 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 02:44:53.265743 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":312,"threads_started":1}
I20260808 02:44:53.267380 7068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:53.349779 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 6882
I20260808 02:44:53.375160 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 7085
Aug 08 02:44:53 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:44:53.390784 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:53.376308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.25.137.195:36895)
0808 02:44:53.376827 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:53.376834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:53.376865 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:44:53.376932 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:53.376939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:53.377217 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:53.377358 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:53.378654 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:53.379624 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:53.380853 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:53.381429 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:53.381535 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:53.385827 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:53.385839 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:53.385853 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:53.385887 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:53.388404 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:53.389043 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:53.389052 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:53.389055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:53.389180 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:53.389512 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:53.389515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:53.389517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:53.389845 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:53.390056 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:53.390284 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 02:44:53.390484 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":244,"threads_started":1}
I20260808 02:44:53.392067 7273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:53.474946 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 7085
I20260808 02:44:53.495849 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 6518
I20260808 02:44:53.497083 6613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:44:53.601723 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 6518
I20260808 02:44:53.626041 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:41401
--webserver_interface=127.25.137.254
--webserver_port=46345
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:43021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:41401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:44:53.965034 7307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:53.965363 7307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:53.965523 7307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:53.972898 7307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:44:53.973055 7307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:53.973150 7307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:53.973242 7307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:44:53.973312 7307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:44:53.983424 7307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:41401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:41401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=46345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:53.984819 7307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:53.986276 7307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:53.998075 7318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:53.998121 7316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:53.998167 7314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:53.999010 7307 server_base.cc:1034] running on GCE node
I20260808 02:44:53.999814 7307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:54.001252 7307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:54.002489 7307 hybrid_clock.cc:648] HybridClock initialized: now 1786157094002429 us; error 48 us; skew 500 ppm
I20260808 02:44:54.002843 7307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:54 dist-test-slave-k5rr krb5kdc[6503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157094, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:54.005954 7307 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:44:54.007431 7307 webserver.cc:460] Webserver started at http://127.25.137.254:46345/ using document root <none> and password file <none>
I20260808 02:44:54.008091 7307 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:54.008224 7307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:54.013664 7307 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:44:54.016812 7329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:54.018165 7307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:44:54.018325 7307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "c42d2b4c116d41f3b2263bc2c6770445"
format_stamp: "Formatted at 2026-08-08 02:44:51 on dist-test-slave-k5rr"
server_key: "b27d2497e6e9e5c451f013d950448f20"
server_key_iv: "6734483920f6851318f27bd3feb0fddd"
server_key_version: "encryptionkey@0"
I20260808 02:44:54.019285 7307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:54.040822 7307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:54.044724 7307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:54.044973 7307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:54.062837 7307 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:41401
I20260808 02:44:54.062883 7411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:41401 every 8 connection(s)
I20260808 02:44:54.064348 7307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:44:54.071188 7412 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:44:54.072820 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 7307
I20260808 02:44:54.073804 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:38765
--local_ip_for_outbound_sockets=127.25.137.193
--tserver_master_addrs=127.25.137.254:41401
--webserver_port=40835
--webserver_interface=127.25.137.193
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:43021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.137.193
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:44:54.075436 7412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445: Bootstrap starting.
I20260808 02:44:54.096454 7412 log.cc:824] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445: Log is configured to *not* fsync() on all Append() calls
I20260808 02:44:54.105572 7412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:44:54.106107 7412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445: Bootstrap complete.
I20260808 02:44:54.115549 7412 raft_consensus.cc:348] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } }
I20260808 02:44:54.116427 7412 raft_consensus.cc:729] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c42d2b4c116d41f3b2263bc2c6770445, State: Initialized, Role: FOLLOWER
I20260808 02:44:54.117100 7412 consensus_queue.cc:260] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } }
I20260808 02:44:54.117352 7412 raft_consensus.cc:388] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:44:54.117470 7412 raft_consensus.cc:482] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:44:54.117623 7412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:44:54.121474 7412 raft_consensus.cc:504] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } }
I20260808 02:44:54.121963 7412 leader_election.cc:302] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c42d2b4c116d41f3b2263bc2c6770445; no voters:
I20260808 02:44:54.122607 7412 leader_election.cc:288] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:44:54.122813 7416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:44:54.123658 7416 raft_consensus.cc:686] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [term 2 LEADER]: Becoming Leader. State: Replica: c42d2b4c116d41f3b2263bc2c6770445, State: Running, Role: LEADER
I20260808 02:44:54.124325 7416 consensus_queue.cc:237] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } }
I20260808 02:44:54.126155 7416 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c42d2b4c116d41f3b2263bc2c6770445. Latest consensus state: current_term: 2 leader_uuid: "c42d2b4c116d41f3b2263bc2c6770445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } } }
I20260808 02:44:54.126456 7416 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:54.127596 7418 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c42d2b4c116d41f3b2263bc2c6770445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42d2b4c116d41f3b2263bc2c6770445" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41401 } } }
I20260808 02:44:54.127918 7418 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [sys.catalog]: This master's current role is: LEADER
I20260808 02:44:54.130987 7412 sys_catalog.cc:564] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:44:54.131618 7421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:44:54.138137 7421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:44:54.139638 7421 catalog_manager.cc:1259] Loaded cluster ID: 4447203d59074cf9995e6a674101e817
I20260808 02:44:54.139743 7421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:44:54.144096 7421 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:44:54.145787 7421 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c42d2b4c116d41f3b2263bc2c6770445: Loaded TSK: 0
I20260808 02:44:54.147382 7421 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:44:54.417830 7414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:54.418133 7414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:54.418279 7414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:54.425843 7414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:54.425962 7414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:54.426115 7414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:44:54.436144 7414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.193.keytab
--principal=oryx/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:38765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=40835
--tserver_master_addrs=127.25.137.254:41401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:54.437526 7414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:54.438815 7414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:54.453471 7445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:54.454559 7414 server_base.cc:1034] running on GCE node
W20260808 02:44:54.454634 7447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:54.454641 7451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:54.455531 7414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:54.456485 7414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:54.457686 7414 hybrid_clock.cc:648] HybridClock initialized: now 1786157094457615 us; error 67 us; skew 500 ppm
I20260808 02:44:54.457991 7414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:54 dist-test-slave-k5rr krb5kdc[6503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157094, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:54.461391 7414 init.cc:377] Logged in from keytab as oryx/127.25.137.193@KRBTEST.COM (short username oryx)
I20260808 02:44:54.462970 7414 webserver.cc:460] Webserver started at http://127.25.137.193:40835/ using document root <none> and password file <none>
I20260808 02:44:54.463641 7414 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:54.463776 7414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:54.469336 7414 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:44:54.472040 7458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:54.473440 7414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:44:54.473588 7414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "5349ab6968e14936aa388879782ec9fa"
format_stamp: "Formatted at 2026-08-08 02:44:51 on dist-test-slave-k5rr"
server_key: "8e21cb34e5dd86485f422fadf19ac4d1"
server_key_iv: "3ef6d83fc515d2106de5b7359d99ae74"
server_key_version: "encryptionkey@0"
I20260808 02:44:54.474527 7414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:54.486960 7414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:54.490844 7414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:54.491099 7414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:54.492112 7414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:54.493465 7414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:54.493561 7414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:54.493664 7414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:54.493739 7414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:54.524958 7414 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:38765
I20260808 02:44:54.525065 7630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:38765 every 8 connection(s)
I20260808 02:44:54.526454 7414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:44:54.532729 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 7414
I20260808 02:44:54.533524 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:36359
--local_ip_for_outbound_sockets=127.25.137.194
--tserver_master_addrs=127.25.137.254:41401
--webserver_port=46765
--webserver_interface=127.25.137.194
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:43021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.137.194
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:44:54 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:54 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:54.545228 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:54.529328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:50209)
0808 02:44:54.530095 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:54.530132 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:54.531827 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:54.533228 (+ 1401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:54.533238 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:54.533959 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:54.534704 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:54.534729 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.536318 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.536324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:54.538002 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:54.538017 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.538970 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.538976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:54.539247 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:54.539929 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:54.539955 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:54.544111 (+ 4156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":325,"threads_started":1}
W20260808 02:44:54.545706 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:54.529266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:50209 (local address 127.25.137.254:41401)
0808 02:44:54.530050 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:54.530065 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:54.530245 (+ 180us) server_negotiation.cc:407] Connection header received
0808 02:44:54.532330 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:54.532349 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:54.532686 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:54.533029 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:54.534907 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.536151 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.538131 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.538740 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.538905 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:54.544420 (+ 5515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:50209: BlockingRecv error: recv got EOF from 127.25.137.193:50209 (error 108)
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":257,"threads_started":1}
W20260808 02:44:54.546025 7631 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:41401 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.137.254:41401: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Aug 08 02:44:54 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:54 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:54.554630 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:54.546655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:50973)
0808 02:44:54.546830 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:54.546845 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:54.546926 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:54.547348 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:54.547353 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:54.547681 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:54.547822 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:54.547830 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.548980 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.548983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:54.549891 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:54.549903 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.550561 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.550567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:54.550726 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:54.551363 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:54.551381 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:54.554363 (+ 2982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:44:54.554939 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:54.546901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:50973 (local address 127.25.137.254:41401)
0808 02:44:54.547076 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:54.547080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:54.547095 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:44:54.547136 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:54.547140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:54.547195 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:54.547276 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:54.547964 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.548836 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.550032 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.550419 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.550527 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:54.554561 (+ 4034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:50973: BlockingRecv error: recv got EOF from 127.25.137.193:50973 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 02:44:54 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:54 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:54.562860 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:54.555304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:54727)
0808 02:44:54.555449 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:54.555467 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:54.555557 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:54.555982 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:54.555987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:54.556280 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:54.556419 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:54.556427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.557436 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.557439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:54.558323 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:54.558334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.558995 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.559000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:54.559161 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:54.559752 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:54.559770 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:54.562580 (+ 2810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 02:44:54.563167 7631 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:44:54.563408 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:54.555550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:54727 (local address 127.25.137.254:41401)
0808 02:44:54.555711 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:54.555715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:54.555729 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:44:54.555781 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:54.555785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:54.555850 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:54.555930 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:54.556553 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.557304 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.558455 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:54.558860 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:54.558962 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:54.562842 (+ 3880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:54727: BlockingRecv error: recv got EOF from 127.25.137.193:54727 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 02:44:54.881403 7635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:54.881769 7635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:54.881939 7635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:54.889210 7635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:54.889340 7635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:54.889508 7635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:44:54.899312 7635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.194.keytab
--principal=oryx/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:36359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=46765
--tserver_master_addrs=127.25.137.254:41401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:54.900652 7635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:54.902083 7635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:54.915200 7642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:54.915305 7646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:54.915472 7650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:54.916323 7635 server_base.cc:1034] running on GCE node
I20260808 02:44:54.917035 7635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:54.918056 7635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:54.919270 7635 hybrid_clock.cc:648] HybridClock initialized: now 1786157094919188 us; error 70 us; skew 500 ppm
I20260808 02:44:54.919608 7635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:54 dist-test-slave-k5rr krb5kdc[6503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157094, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:54.923089 7635 init.cc:377] Logged in from keytab as oryx/127.25.137.194@KRBTEST.COM (short username oryx)
I20260808 02:44:54.924674 7635 webserver.cc:460] Webserver started at http://127.25.137.194:46765/ using document root <none> and password file <none>
I20260808 02:44:54.925434 7635 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:54.925583 7635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:54.931161 7635 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 02:44:54.934077 7659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:54.935472 7635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:44:54.935637 7635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "21acbbc1e9954ca5a1ced11cc6f57d4e"
format_stamp: "Formatted at 2026-08-08 02:44:52 on dist-test-slave-k5rr"
server_key: "a2254445d279751efd970a64bd62a8e9"
server_key_iv: "e2a63080f943af5f60ff63ed671c9ba7"
server_key_version: "encryptionkey@0"
I20260808 02:44:54.936609 7635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:54.953068 7635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:54.957322 7635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:54.957581 7635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:54.958676 7635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:54.960021 7635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:54.960119 7635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:54.960232 7635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:54.960321 7635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:54.992401 7635 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:36359
I20260808 02:44:54.992479 7839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:36359 every 8 connection(s)
I20260808 02:44:54.993968 7635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:44:55.003917 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 7635
I20260808 02:44:55.004638 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:36895
--local_ip_for_outbound_sockets=127.25.137.195
--tserver_master_addrs=127.25.137.254:41401
--webserver_port=40291
--webserver_interface=127.25.137.195
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:43021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.137.195
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:44:55.009521 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:54.996446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:37589 (local address 127.25.137.254:41401)
0808 02:44:54.996616 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:54.996621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:54.997863 (+ 1242us) server_negotiation.cc:407] Connection header received
0808 02:44:54.999632 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:54.999636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:54.999706 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:54.999795 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:55.001325 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.002047 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.003552 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.004150 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.004276 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.009281 (+ 5005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:37589: BlockingRecv error: recv got EOF from 127.25.137.194:37589 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:44:55.010125 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:54.996971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:37589)
0808 02:44:54.997695 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:54.997731 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:54.999381 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:55.000014 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:55.000023 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:55.000551 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:55.001160 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.001180 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.002180 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.002183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.003417 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.003430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.004354 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.004359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.004581 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.005390 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:55.005414 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:55.009032 (+ 3618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":271,"threads_started":1}
W20260808 02:44:55.010808 7840 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:41401 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.137.254:41401: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:55.019016 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:55.011373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:37311)
0808 02:44:55.011550 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:55.011569 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:55.011667 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:55.012023 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:55.012027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:55.012349 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:55.012483 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.012492 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.013527 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.013532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.014485 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.014497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.015153 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.015159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.015327 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.015846 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:55.015863 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:55.018772 (+ 2909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 02:44:55.019163 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:55.011567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:37311 (local address 127.25.137.254:41401)
0808 02:44:55.011758 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:55.011763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:55.011780 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:44:55.011833 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:55.011836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:55.011893 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:55.011973 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:55.012620 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.013397 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.014611 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.015012 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.015125 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.018957 (+ 3832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:37311: BlockingRecv error: recv got EOF from 127.25.137.194:37311 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:55.027199 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:55.019748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:45063)
0808 02:44:55.019905 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:55.019923 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:55.020013 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:55.020287 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:55.020292 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:55.020597 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:55.020732 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.020740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.021737 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.021743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.022666 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.022677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.023433 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.023439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.023634 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.024191 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:55.024208 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:55.026908 (+ 2700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 02:44:55.027454 7840 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:44:55.027706 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:55.019871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:45063 (local address 127.25.137.254:41401)
0808 02:44:55.020023 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:55.020028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:55.020045 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:44:55.020109 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:55.020114 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:55.020169 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:55.020245 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:55.020866 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.021602 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.022795 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.023298 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.023402 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.027157 (+ 3755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:45063: BlockingRecv error: recv got EOF from 127.25.137.194:45063 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 02:44:55.339772 7845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:44:55.340076 7845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:44:55.340224 7845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:44:55.347397 7845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:44:55.347525 7845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:44:55.347678 7845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:44:55.357386 7845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/krb5kdc/oryx_127.25.137.195.keytab
--principal=oryx/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:36895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=40291
--tserver_master_addrs=127.25.137.254:41401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:44:55.358726 7845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:44:55.360045 7845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:44:55.372866 7851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:55.373147 7852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:44:55.373323 7855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:44:55.374132 7845 server_base.cc:1034] running on GCE node
I20260808 02:44:55.374825 7845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:44:55.375767 7845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:44:55.376996 7845 hybrid_clock.cc:648] HybridClock initialized: now 1786157095376944 us; error 35 us; skew 500 ppm
I20260808 02:44:55.377341 7845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157095, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:44:55.380501 7845 init.cc:377] Logged in from keytab as oryx/127.25.137.195@KRBTEST.COM (short username oryx)
I20260808 02:44:55.382046 7845 webserver.cc:460] Webserver started at http://127.25.137.195:40291/ using document root <none> and password file <none>
I20260808 02:44:55.382736 7845 fs_manager.cc:359] Metadata directory not provided
I20260808 02:44:55.382874 7845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:44:55.388257 7845 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 02:44:55.390949 7867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:55.392304 7845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:44:55.392463 7845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "8930f6ce5c9b4ebe965d90b734d2bb85"
format_stamp: "Formatted at 2026-08-08 02:44:52 on dist-test-slave-k5rr"
server_key: "4ac52da28848aa6aa02a2cbc7b61223d"
server_key_iv: "ee0350c03fde8b65cafc1eeb3d595e1a"
server_key_version: "encryptionkey@0"
I20260808 02:44:55.393524 7845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:44:55.420470 7845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:44:55.424885 7845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:44:55.425176 7845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:44:55.426163 7845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:44:55.427543 7845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:44:55.427644 7845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:55.427762 7845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:44:55.427842 7845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:44:55.460057 7845 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:36895
I20260808 02:44:55.460125 8043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:36895 every 8 connection(s)
I20260808 02:44:55.461578 7845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:44:55.464008 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 7845
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:44:55.480051 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:55.464375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:46513 (local address 127.25.137.254:41401)
0808 02:44:55.464590 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:55.464595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:55.466759 (+ 2164us) server_negotiation.cc:407] Connection header received
0808 02:44:55.468863 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:55.468867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:55.468936 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:55.469067 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:55.470950 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.471708 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.473394 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.473798 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.473908 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.479553 (+ 5645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:46513: BlockingRecv error: recv got EOF from 127.25.137.195:46513 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
I20260808 02:44:55.480525 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:55.464992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:46513)
0808 02:44:55.466606 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:55.466650 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:55.468500 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:55.469325 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:55.469337 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:55.469996 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:55.470775 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.470797 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.471856 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.471861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.473243 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.473255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.474029 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.474037 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.474258 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.474841 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:55.474864 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:55.479210 (+ 4346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1204,"thread_start_us":937,"threads_started":1}
W20260808 02:44:55.481537 8044 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:41401 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.137.254:41401: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
I20260808 02:44:55.483100 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:55.465254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42326 (local address 127.25.137.254:41401)
0808 02:44:55.466027 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:55.466033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:55.466050 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:44:55.466118 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:55.466123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:55.466203 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:55.466317 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:55.468146 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.469238 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.471076 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.471576 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.471709 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.474579 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:44:55.474588 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:44:55.474604 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:44:55.474642 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:44:55.478150 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:55.479114 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:55.479124 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:55.479132 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:55.479425 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:44:55.479891 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:44:55.479896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:44:55.479899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:44:55.480685 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:44:55.480993 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 02:44:55.482312 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0808 02:44:55.482663 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":520,"threads_started":1}
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:55.490321 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:55.482249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:59765)
0808 02:44:55.482649 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:55.482666 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:55.482816 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:55.483245 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:55.483277 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:55.483713 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:55.483890 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.483902 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.485073 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.485078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.485954 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.485968 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.486849 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.486854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.486974 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.487460 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:55.487477 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:55.490070 (+ 2593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277}
W20260808 02:44:55.490501 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:55.482334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:59765 (local address 127.25.137.254:41401)
0808 02:44:55.482504 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:55.482510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:55.482724 (+ 214us) server_negotiation.cc:407] Connection header received
0808 02:44:55.482974 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:55.482980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:55.483063 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:55.483521 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:55.484112 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.484912 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.486119 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.486569 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.486662 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.490276 (+ 3614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:59765: BlockingRecv error: recv got EOF from 127.25.137.195:59765 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:55.498059 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:55.491000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:34569)
0808 02:44:55.491151 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:55.491165 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:55.491249 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:55.491621 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:55.491625 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:55.491903 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:55.492039 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.492047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.493033 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.493039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.493938 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.493950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.494593 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.494598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.494754 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.495236 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:55.495253 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:55.497746 (+ 2493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 02:44:55.498385 8044 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:44:55.498543 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:55.491123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:34569 (local address 127.25.137.254:41401)
0808 02:44:55.491354 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:55.491358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:55.491397 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:44:55.491433 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:55.491436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:55.491495 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:55.491571 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:55.492171 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.492859 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.494074 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.494455 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.494560 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.497956 (+ 3396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:34569: BlockingRecv error: recv got EOF from 127.25.137.195:34569 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:55 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:55.573074 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:55.563768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:42789)
0808 02:44:55.564362 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:55.564378 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:55.564515 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:55.564901 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:55.564907 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:55.565432 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:55.565630 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.565640 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.566655 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.566659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.567988 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:55.568003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.568740 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.568746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:55.568900 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.569523 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:55.569541 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:55.572728 (+ 3187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":302,"threads_started":1}
W20260808 02:44:55.573215 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:55.563905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:42789 (local address 127.25.137.254:41401)
0808 02:44:55.564106 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:55.564112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:55.564473 (+ 361us) server_negotiation.cc:407] Connection header received
0808 02:44:55.564648 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:55.564652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:55.564716 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:55.564793 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:55.565780 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.566506 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.568129 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:55.568558 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:55.568664 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:55.572976 (+ 4312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:42789: BlockingRecv error: recv got EOF from 127.25.137.193:42789 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 02:44:56 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:56 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:56.036491 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:56.028179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:36375)
0808 02:44:56.028727 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:56.028743 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:56.028865 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:56.029172 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:56.029176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:56.029642 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:56.029874 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:56.029884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.030828 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.030858 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:56.031853 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:56.031869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.032645 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.032650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:56.032782 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:56.033428 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:56.033446 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:56.036243 (+ 2797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":300,"threads_started":1}
W20260808 02:44:56.036690 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:56.028220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:36375 (local address 127.25.137.254:41401)
0808 02:44:56.028447 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:56.028455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:56.028819 (+ 364us) server_negotiation.cc:407] Connection header received
0808 02:44:56.028949 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:56.028952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:56.029047 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:56.029129 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:56.029992 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.030694 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.031983 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.032460 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.032553 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:56.036456 (+ 3903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:36375: BlockingRecv error: recv got EOF from 127.25.137.194:36375 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 02:44:56 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:56 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:56.513233 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:56.498970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:55453)
0808 02:44:56.499491 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:56.499507 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:56.499639 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:56.499982 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:56.499987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:56.500497 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:56.500693 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:56.500702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.501770 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.501774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:56.502728 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:56.502741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.503467 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.503474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:56.503633 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:56.510070 (+ 6437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:56.510090 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:56.512932 (+ 2842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":285,"threads_started":1}
W20260808 02:44:56.513408 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:56.499045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:55453 (local address 127.25.137.254:41401)
0808 02:44:56.499233 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:56.499240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:56.499601 (+ 361us) server_negotiation.cc:407] Connection header received
0808 02:44:56.499755 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:56.499761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:56.499835 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:56.499921 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:56.500847 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.501621 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.502877 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.503279 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.503396 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:56.513193 (+ 9797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:55453: BlockingRecv error: recv got EOF from 127.25.137.195:55453 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 02:44:56 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:56 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:56.582608 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:56.574012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:58301)
0808 02:44:56.574498 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:56.574515 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:56.574647 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:56.574934 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:56.574937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:56.575345 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:56.575536 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:56.575547 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.576578 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.576582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:56.577679 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:56.577696 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.578638 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.578644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:56.578831 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:56.579418 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:56.579436 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:56.582275 (+ 2839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":260,"threads_started":1}
W20260808 02:44:56.582804 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:56.574078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:58301 (local address 127.25.137.254:41401)
0808 02:44:56.574312 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:56.574318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:56.574642 (+ 324us) server_negotiation.cc:407] Connection header received
0808 02:44:56.574749 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:56.574753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:56.574817 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:56.574901 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:56.575678 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.576458 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.577868 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:56.578454 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:56.578576 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:56.582550 (+ 3974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:58301: BlockingRecv error: recv got EOF from 127.25.137.193:58301 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 02:44:57 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:57 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:57.046010 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:57.037425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:57687)
0808 02:44:57.037959 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:57.037976 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:57.038107 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:57.038419 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:57.038422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:57.038860 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:57.039036 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:57.039046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.040167 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.040171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:57.041209 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:57.041223 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.041907 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.041911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:57.042022 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:57.042575 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:57.042593 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:57.045707 (+ 3114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":282,"threads_started":1}
W20260808 02:44:57.046240 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:57.037509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:57687 (local address 127.25.137.254:41401)
0808 02:44:57.037757 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:57.037763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:57.038055 (+ 292us) server_negotiation.cc:407] Connection header received
0808 02:44:57.038201 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:57.038204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:57.038286 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:57.038386 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:57.039193 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.040036 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.041337 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.041748 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.041856 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:57.046000 (+ 4144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:57687: BlockingRecv error: recv got EOF from 127.25.137.194:57687 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 02:44:57 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:57 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:57.522496 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:57.514247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:57521)
0808 02:44:57.514745 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:57.514765 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:57.514907 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:57.515247 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:57.515252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:57.515684 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:57.515859 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:57.515869 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.516882 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.516888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:57.517869 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:57.517882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.518597 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.518603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:57.518768 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:57.519383 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:57.519400 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:57.522161 (+ 2761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":255,"threads_started":1}
W20260808 02:44:57.522626 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:57.514545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:57521 (local address 127.25.137.254:41401)
0808 02:44:57.514743 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:57.514749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:57.514866 (+ 117us) server_negotiation.cc:407] Connection header received
0808 02:44:57.515028 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:57.515033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:57.515095 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:57.515175 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:57.516003 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.516732 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.518008 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.518440 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.518548 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:57.522379 (+ 3831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:57521: BlockingRecv error: recv got EOF from 127.25.137.195:57521 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 02:44:57 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:57 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:57.592852 8061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:57.583678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:46605)
0808 02:44:57.584208 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:57.584225 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:57.584374 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:57.584692 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:57.584713 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:57.585139 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:57.585319 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:57.585329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.586718 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.586725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:57.588185 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:57.588203 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.588930 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.588934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:57.589130 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:57.589698 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:57.589717 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:57.592481 (+ 2764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":266,"threads_started":1}
W20260808 02:44:57.592945 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:57.583719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:46605 (local address 127.25.137.254:41401)
0808 02:44:57.583940 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:57.583947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:57.584317 (+ 370us) server_negotiation.cc:407] Connection header received
0808 02:44:57.584503 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:57.584507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:57.584574 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:57.584651 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:57.585466 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.586553 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.588388 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:57.588798 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:57.588906 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:57.592713 (+ 3807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:46605: BlockingRecv error: recv got EOF from 127.25.137.193:46605 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 08 02:44:58 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:58 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:58.055429 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:58.047146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:32959)
0808 02:44:58.047667 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:58.047683 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:58.047839 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:58.048126 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:58.048129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:58.048492 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:58.048656 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:58.048665 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.049789 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.049793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:58.050720 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:58.050733 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.051416 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.051420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:58.051571 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:58.052154 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:58.052173 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:58.055048 (+ 2875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":286,"threads_started":1}
W20260808 02:44:58.055783 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:58.047245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:32959 (local address 127.25.137.254:41401)
0808 02:44:58.047415 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:58.047419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:58.047751 (+ 332us) server_negotiation.cc:407] Connection header received
0808 02:44:58.047941 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:58.047944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:58.048009 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:58.048090 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:58.048771 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.049609 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.050870 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.051278 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.051379 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:58.055447 (+ 4068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:32959: BlockingRecv error: recv got EOF from 127.25.137.194:32959 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 02:44:58 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:58 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:58.533128 8063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:58.523560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:45939)
0808 02:44:58.524258 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:58.524281 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:58.524411 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:58.524767 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:58.524773 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:58.525336 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:58.525569 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:58.525583 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.526713 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.526720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:58.528177 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:58.528192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.528909 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.528914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:58.529083 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:58.529698 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:58.529718 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:58.532741 (+ 3023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":411,"threads_started":1}
W20260808 02:44:58.533267 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:58.523678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:45939 (local address 127.25.137.254:41401)
0808 02:44:58.523891 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:58.523896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:58.524364 (+ 468us) server_negotiation.cc:407] Connection header received
0808 02:44:58.524513 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:58.524517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:58.524583 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:58.524665 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:58.525751 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.526544 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.528322 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.528742 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.528843 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:58.532981 (+ 4138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:45939: BlockingRecv error: recv got EOF from 127.25.137.195:45939 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 02:44:58 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:58 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:58.602579 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:58.593968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:46901)
0808 02:44:58.594475 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:58.594495 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:58.594632 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:58.594948 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:58.594951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:58.595338 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:58.595502 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:58.595511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.596726 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.596730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:58.597786 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:58.597836 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.598617 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.598621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:58.598749 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:58.599366 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:58.599384 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:58.602347 (+ 2963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":257,"threads_started":1}
W20260808 02:44:58.602856 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:58.594055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:46901 (local address 127.25.137.254:41401)
0808 02:44:58.594331 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:58.594338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:58.594588 (+ 250us) server_negotiation.cc:407] Connection header received
0808 02:44:58.594743 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:58.594747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:58.594816 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:58.594900 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:58.595647 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.596560 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.597966 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:58.598470 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:58.598571 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:58.602554 (+ 3983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:46901: BlockingRecv error: recv got EOF from 127.25.137.193:46901 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 08 02:44:59 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:59 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:59.065284 8065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:59.056516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:34375)
0808 02:44:59.057080 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:59.057103 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:59.057223 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:59.057580 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:59.057583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:59.057964 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:59.058116 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:59.058124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.059146 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.059149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:59.060113 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:59.060124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.060893 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.060897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:59.061055 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:59.061621 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:59.061639 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:59.064857 (+ 3218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":289,"threads_started":1}
W20260808 02:44:59.065568 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:59.056574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:34375 (local address 127.25.137.254:41401)
0808 02:44:59.056800 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:59.056809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:59.057195 (+ 386us) server_negotiation.cc:407] Connection header received
0808 02:44:59.057339 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:59.057346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:59.057437 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:59.057536 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:59.058272 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.059038 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.060270 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.060749 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.060859 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:59.065252 (+ 4393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:34375: BlockingRecv error: recv got EOF from 127.25.137.194:34375 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 02:44:59 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:59 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:59.542843 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:59.534164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:47925)
0808 02:44:59.534711 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:59.534727 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:59.534904 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:59.535259 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:59.535264 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:59.535656 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:59.535822 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:59.535832 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.536889 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.536893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:59.537984 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:59.537998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.538698 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.538704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:59.538879 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:59.539470 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:59.539487 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:59.542530 (+ 3043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":323,"threads_started":1}
W20260808 02:44:59.542997 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:59.534249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:47925 (local address 127.25.137.254:41401)
0808 02:44:59.534482 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:59.534488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:59.534810 (+ 322us) server_negotiation.cc:407] Connection header received
0808 02:44:59.535028 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:59.535034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:59.535100 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:59.535183 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:59.535972 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.536748 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.538125 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.538555 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.538664 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:59.542738 (+ 4074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:47925: BlockingRecv error: recv got EOF from 127.25.137.195:47925 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 02:44:59 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:44:59 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:44:59.611517 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:44:59.603496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:53391)
0808 02:44:59.604009 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 02:44:59.604025 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:44:59.604139 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:44:59.604421 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:44:59.604424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:44:59.604853 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:44:59.605061 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:59.605073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.606118 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.606122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:59.607038 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:44:59.607051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.607725 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.607729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:44:59.607887 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:59.608427 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:44:59.608443 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:44:59.611239 (+ 2796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":228,"threads_started":1}
W20260808 02:44:59.611773 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:44:59.603549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:53391 (local address 127.25.137.254:41401)
0808 02:44:59.603814 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 02:44:59.603822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:44:59.604096 (+ 274us) server_negotiation.cc:407] Connection header received
0808 02:44:59.604228 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:44:59.604233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:44:59.604309 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:44:59.604389 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:44:59.605219 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.605993 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.607162 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:44:59.607591 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:44:59.607687 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:44:59.611505 (+ 3818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:53391: BlockingRecv error: recv got EOF from 127.25.137.193:53391 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 02:45:00 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:00 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:00.078620 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:00.069612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:37447)
0808 02:45:00.070175 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:00.070194 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:00.070352 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:00.070703 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:00.070709 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:00.071294 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:00.071558 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:00.071576 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.072757 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.072763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:00.073853 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:00.073869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.074558 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.074562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:00.074694 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:00.075290 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:00.075309 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:00.078370 (+ 3061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":278,"threads_started":1}
W20260808 02:45:00.078919 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:00.069760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:37447 (local address 127.25.137.254:41401)
0808 02:45:00.069916 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:00.069921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:00.070279 (+ 358us) server_negotiation.cc:407] Connection header received
0808 02:45:00.070472 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:00.070477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:00.070541 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:00.070650 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:00.071722 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.072621 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.074008 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.074428 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.074522 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:00.078579 (+ 4057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:37447: BlockingRecv error: recv got EOF from 127.25.137.194:37447 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 02:45:00 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:00 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:00.553246 8070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:00.543998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:39225)
0808 02:45:00.544608 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:00.544625 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:00.544821 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:00.545231 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:00.545236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:00.545695 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:00.545887 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:00.545898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.546989 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.546994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:00.547989 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:00.548002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.548839 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.548845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:00.549071 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:00.549678 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:00.549696 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:00.552942 (+ 3246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":310,"threads_started":1}
W20260808 02:45:00.553473 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:00.544139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:39225 (local address 127.25.137.254:41401)
0808 02:45:00.544403 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:00.544411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:00.544723 (+ 312us) server_negotiation.cc:407] Connection header received
0808 02:45:00.544974 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:00.544981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:00.545074 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:00.545169 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:00.546064 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.546830 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.548176 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.548665 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.548799 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:00.553214 (+ 4415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:39225: BlockingRecv error: recv got EOF from 127.25.137.195:39225 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 08 02:45:00 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:00 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:00.620447 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:00.612366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:58033)
0808 02:45:00.612865 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:00.612882 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:00.612997 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:00.613325 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:00.613327 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:00.613711 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:00.613913 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:00.613923 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.615011 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.615015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:00.615954 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:00.615966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.616696 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.616700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:00.616876 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:00.617498 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:00.617515 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:00.620222 (+ 2707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":269,"threads_started":1}
W20260808 02:45:00.620730 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:00.612431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:58033 (local address 127.25.137.254:41401)
0808 02:45:00.612644 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:00.612652 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:00.612960 (+ 308us) server_negotiation.cc:407] Connection header received
0808 02:45:00.613148 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:00.613152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:00.613217 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:00.613314 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:00.614081 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.614871 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.616092 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:00.616547 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:00.616644 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:00.620459 (+ 3815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:58033: BlockingRecv error: recv got EOF from 127.25.137.193:58033 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 02:45:01 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:01 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:01.087610 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:01.079672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:34557)
0808 02:45:01.080184 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:01.080200 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:01.080314 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:01.080582 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:01.080585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:01.081084 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:01.081251 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:01.081261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.082254 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.082258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:01.083160 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:01.083172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.083880 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.083885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:01.084010 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:01.084610 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:01.084629 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:01.087384 (+ 2755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":287,"threads_started":1}
W20260808 02:45:01.087879 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:01.079736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:34557 (local address 127.25.137.254:41401)
0808 02:45:01.079936 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:01.079944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:01.080280 (+ 336us) server_negotiation.cc:407] Connection header received
0808 02:45:01.080406 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:01.080409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:01.080474 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:01.080550 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:01.081410 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.082125 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.083304 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.083717 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.083814 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:01.087624 (+ 3810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:34557: BlockingRecv error: recv got EOF from 127.25.137.194:34557 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 02:45:01 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:01 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:01.565496 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:01.554205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:48031)
0808 02:45:01.554769 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:01.554785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:01.554956 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:01.555291 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:01.555296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:01.555718 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:01.555888 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:01.555899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.557082 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.557086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:01.558353 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:01.558370 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.559204 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.559208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:01.559347 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:01.562386 (+ 3039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:01.562407 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:01.565189 (+ 2782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":330,"threads_started":1}
W20260808 02:45:01.565665 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:01.554319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:48031 (local address 127.25.137.254:41401)
0808 02:45:01.554510 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:01.554517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:01.554908 (+ 391us) server_negotiation.cc:407] Connection header received
0808 02:45:01.555075 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:01.555081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:01.555153 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:01.555236 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:01.556037 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.556925 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.558504 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.559080 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.559174 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:01.565428 (+ 6254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:48031: BlockingRecv error: recv got EOF from 127.25.137.195:48031 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 02:45:01 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:01 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:01.630304 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:01.621438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:47021)
0808 02:45:01.622055 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:01.622072 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:01.622215 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:01.622578 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:01.622582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:01.623066 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:01.623251 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:01.623261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.624261 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.624265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:01.625218 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:01.625230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.625947 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.625953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:01.626105 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:01.626635 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:01.626652 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:01.629961 (+ 3309us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":241,"threads_started":1}
W20260808 02:45:01.630414 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:01.621502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:47021 (local address 127.25.137.254:41401)
0808 02:45:01.621659 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:01.621666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:01.622160 (+ 494us) server_negotiation.cc:407] Connection header received
0808 02:45:01.622369 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:01.622373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:01.622439 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:01.622521 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:01.623394 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.624100 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.625369 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:01.625791 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:01.625904 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:01.630194 (+ 4290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:47021: BlockingRecv error: recv got EOF from 127.25.137.193:47021 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 02:45:02 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:02 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:02.096766 8075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:02.088529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:34451)
0808 02:45:02.089101 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:02.089118 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:02.089233 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:02.089510 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:02.089513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:02.089966 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:02.090135 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:02.090144 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.091142 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.091146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:02.092101 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:02.092113 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.092807 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.092812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:02.092936 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:02.093499 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:02.093516 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:02.096462 (+ 2946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":246,"threads_started":1}
W20260808 02:45:02.096990 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:02.088602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:34451 (local address 127.25.137.254:41401)
0808 02:45:02.088827 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:02.088835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:02.089186 (+ 351us) server_negotiation.cc:407] Connection header received
0808 02:45:02.089333 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:02.089336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:02.089401 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:02.089480 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:02.090251 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.091012 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.092212 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.092630 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.092718 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:02.096688 (+ 3970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:34451: BlockingRecv error: recv got EOF from 127.25.137.194:34451 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 02:45:02 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:02 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:02.574827 8081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:02.566474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:56541)
0808 02:45:02.567114 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:02.567131 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:02.567246 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:02.567560 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:02.567563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:02.568019 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:02.568196 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:02.568206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.569376 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.569380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:02.570302 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:02.570315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.571018 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.571022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:02.571135 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:02.571718 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:02.571737 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:02.574499 (+ 2762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":327,"threads_started":1}
W20260808 02:45:02.574921 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:02.566557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:56541 (local address 127.25.137.254:41401)
0808 02:45:02.566785 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:02.566794 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:02.567221 (+ 427us) server_negotiation.cc:407] Connection header received
0808 02:45:02.567349 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:02.567353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:02.567434 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:02.567528 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:02.568340 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.569224 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.570427 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.570882 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.570996 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:02.574720 (+ 3724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:56541: BlockingRecv error: recv got EOF from 127.25.137.195:56541 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 08 02:45:02 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:02 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:02.640054 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:02.631343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:60771)
0808 02:45:02.631954 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:02.631972 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:02.632094 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:02.632431 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:02.632435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:02.632916 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:02.633107 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:02.633117 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.634163 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.634167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:02.635234 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:02.635248 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.636208 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.636216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:02.636378 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:02.637075 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:02.637094 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:02.639776 (+ 2682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":298,"threads_started":1}
W20260808 02:45:02.640218 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:02.631507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:60771 (local address 127.25.137.254:41401)
0808 02:45:02.631667 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:02.631672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:02.632046 (+ 374us) server_negotiation.cc:407] Connection header received
0808 02:45:02.632185 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:02.632189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:02.632270 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:02.632360 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:02.633229 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.634031 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.635366 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:02.636016 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:02.636141 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:02.639996 (+ 3855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:60771: BlockingRecv error: recv got EOF from 127.25.137.193:60771 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 02:45:03 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:03 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:03.106276 8084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:03.097898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:43523)
0808 02:45:03.098484 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:03.098501 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:03.098614 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:03.098886 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:03.098890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:03.099281 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:03.099463 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:03.099472 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.100654 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.100658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:03.101733 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:03.101749 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.102448 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.102452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:03.102563 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:03.103132 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:03.103149 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:03.106030 (+ 2881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":271,"threads_started":1}
W20260808 02:45:03.106477 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:03.097947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:43523 (local address 127.25.137.254:41401)
0808 02:45:03.098123 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:03.098128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:03.098570 (+ 442us) server_negotiation.cc:407] Connection header received
0808 02:45:03.098695 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:03.098699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:03.098762 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:03.098840 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:03.099568 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.100468 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.101870 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.102297 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.102398 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:03.106239 (+ 3841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:43523: BlockingRecv error: recv got EOF from 127.25.137.194:43523 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 02:45:03 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:03 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:03.584923 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:03.575906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:55927)
0808 02:45:03.576656 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:03.576678 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:03.576914 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:03.577428 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:03.577433 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:03.578043 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:03.578210 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:03.578220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.579355 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.579359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:03.580300 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:03.580313 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.581073 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.581081 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:03.581196 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:03.581823 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:03.581841 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:03.584569 (+ 2728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":446,"threads_started":1}
W20260808 02:45:03.585072 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:03.576026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:55927 (local address 127.25.137.254:41401)
0808 02:45:03.576202 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:03.576209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:03.576845 (+ 636us) server_negotiation.cc:407] Connection header received
0808 02:45:03.577079 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:03.577084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:03.577164 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:03.577283 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:03.578415 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.579204 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.580433 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.580873 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.580965 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:03.584809 (+ 3844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:55927: BlockingRecv error: recv got EOF from 127.25.137.195:55927 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 02:45:03 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:03 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:03.649710 8086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:03.640979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:58137)
0808 02:45:03.641524 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:03.641540 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:03.641692 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:03.642092 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:03.642097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:03.642544 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:03.642715 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:03.642725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.643724 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.643729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:03.644686 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:03.644698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.645495 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.645501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:03.645662 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:03.646215 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:03.646234 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:03.649393 (+ 3159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":284,"threads_started":1}
W20260808 02:45:03.649870 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:03.641057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:58137 (local address 127.25.137.254:41401)
0808 02:45:03.641252 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:03.641259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:03.641631 (+ 372us) server_negotiation.cc:407] Connection header received
0808 02:45:03.641855 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:03.641859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:03.641939 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:03.642025 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:03.642873 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.643588 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.644840 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:03.645339 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:03.645452 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:03.649595 (+ 4143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:58137: BlockingRecv error: recv got EOF from 127.25.137.193:58137 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 02:45:04 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:04 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:04.115518 8088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:04.107227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:32889)
0808 02:45:04.107732 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:04.107749 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:04.107919 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:04.108242 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:04.108247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:04.108671 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:04.108854 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:04.108864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.109929 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.109933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:04.110901 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:04.110914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.111663 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.111669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:04.111842 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:04.112405 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:04.112423 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:04.115245 (+ 2822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":262,"threads_started":1}
W20260808 02:45:04.115666 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:04.107304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:32889 (local address 127.25.137.254:41401)
0808 02:45:04.107495 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:04.107501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:04.107834 (+ 333us) server_negotiation.cc:407] Connection header received
0808 02:45:04.108037 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:04.108041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:04.108106 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:04.108187 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:04.109001 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.109791 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.111051 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.111510 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.111621 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:04.115461 (+ 3840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:32889: BlockingRecv error: recv got EOF from 127.25.137.194:32889 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 02:45:04 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:04 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:04.594632 8090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:04.585887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:37099)
0808 02:45:04.586426 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:04.586442 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:04.586559 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:04.586943 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:04.586948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:04.587333 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:04.587503 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:04.587512 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.588535 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.588539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:04.589580 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:04.589593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.590327 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.590333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:04.590477 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:04.591144 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:04.591166 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:04.594377 (+ 3211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":275,"threads_started":1}
W20260808 02:45:04.594720 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:04.585962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:37099 (local address 127.25.137.254:41401)
0808 02:45:04.586126 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:04.586131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:04.586511 (+ 380us) server_negotiation.cc:407] Connection header received
0808 02:45:04.586677 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:04.586680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:04.586753 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:04.586859 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:04.587642 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.588399 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.589711 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.590145 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.590250 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:04.594542 (+ 4292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:37099: BlockingRecv error: recv got EOF from 127.25.137.195:37099 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 02:45:04 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:04 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:04.659150 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:04.650686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:52149)
0808 02:45:04.651242 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:04.651259 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:04.651402 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:04.651765 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:04.651770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:04.652176 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:04.652393 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:04.652410 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.653648 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.653653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:04.654582 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:04.654594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.655324 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.655331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:04.655480 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:04.656085 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:04.656102 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:04.658908 (+ 2806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":316,"threads_started":1}
W20260808 02:45:04.659339 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:04.650857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:52149 (local address 127.25.137.254:41401)
0808 02:45:04.651072 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:04.651079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:04.651355 (+ 276us) server_negotiation.cc:407] Connection header received
0808 02:45:04.651538 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:04.651542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:04.651622 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:04.651706 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:04.652556 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.653511 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.654725 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:04.655151 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:04.655259 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:04.659100 (+ 3841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:52149: BlockingRecv error: recv got EOF from 127.25.137.193:52149 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 02:45:05 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:05 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:05.125492 8092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:05.116719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:43117)
0808 02:45:05.117478 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:05.117497 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:05.117646 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:05.118088 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:05.118093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:05.118600 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:05.118809 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:05.118823 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.119903 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.119908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:05.120854 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:05.120867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.121604 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.121609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:05.121726 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:05.122329 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:05.122347 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:05.125234 (+ 2887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":497,"threads_started":1}
W20260808 02:45:05.125677 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:05.116824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:43117 (local address 127.25.137.254:41401)
0808 02:45:05.117052 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:05.117059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:05.117597 (+ 538us) server_negotiation.cc:407] Connection header received
0808 02:45:05.117783 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:05.117789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:05.117884 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:05.117983 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:05.118984 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.119766 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.120993 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.121462 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.121553 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:05.125443 (+ 3890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:43117: BlockingRecv error: recv got EOF from 127.25.137.194:43117 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 02:45:05 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:05 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:05.604566 8093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:05.595616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:55457)
0808 02:45:05.596132 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:05.596147 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:05.596275 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:05.596646 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:05.596650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:05.597246 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:05.597414 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:05.597424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.598643 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.598648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:05.599601 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:05.599613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.600338 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.600342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:05.600497 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:05.601126 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:05.601144 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:05.604194 (+ 3050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":280,"threads_started":1}
W20260808 02:45:05.604686 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:05.595816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:55457 (local address 127.25.137.254:41401)
0808 02:45:05.596033 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:05.596039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:05.596226 (+ 187us) server_negotiation.cc:407] Connection header received
0808 02:45:05.596418 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:05.596422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:05.596484 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:05.596592 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:05.597565 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.598451 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.599741 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.600207 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.600301 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:05.604477 (+ 4176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:55457: BlockingRecv error: recv got EOF from 127.25.137.195:55457 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 02:45:05 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:05 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:05.668962 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:05.660049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:38609)
0808 02:45:05.660521 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:05.660538 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:05.660653 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:05.660952 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:05.660954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:05.661417 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:05.661602 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:05.661612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.663049 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.663054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:05.664095 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:05.664109 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.664836 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.664842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:05.664969 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:05.665533 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:05.665551 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:05.668710 (+ 3159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":263,"threads_started":1}
W20260808 02:45:05.669348 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:05.660157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:38609 (local address 127.25.137.254:41401)
0808 02:45:05.660424 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:05.660431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:05.660612 (+ 181us) server_negotiation.cc:407] Connection header received
0808 02:45:05.660741 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:05.660745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:05.660830 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:05.660910 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:05.661771 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.662877 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.664224 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:05.664667 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:05.664756 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:05.668968 (+ 4212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:38609: BlockingRecv error: recv got EOF from 127.25.137.193:38609 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 08 02:45:06 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:06 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:06.135591 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:06.126388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:50373)
0808 02:45:06.126935 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:06.126954 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:06.127088 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:06.127448 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:06.127453 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:06.127886 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:06.128071 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:06.128080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.129228 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.129233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:06.130387 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:06.130403 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.131398 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.131404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:06.131627 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:06.132226 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:06.132245 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:06.135337 (+ 3092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":291,"threads_started":1}
W20260808 02:45:06.135797 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:06.126526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:50373 (local address 127.25.137.254:41401)
0808 02:45:06.126740 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:06.126746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:06.127037 (+ 291us) server_negotiation.cc:407] Connection header received
0808 02:45:06.127187 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:06.127191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:06.127258 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:06.127358 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:06.128223 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.129082 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.130539 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.131193 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.131305 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:06.135547 (+ 4242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:50373: BlockingRecv error: recv got EOF from 127.25.137.194:50373 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 02:45:06 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:06 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:06.618554 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:06.605478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:60101)
0808 02:45:06.606092 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:06.606108 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:06.606250 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:06.606637 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:06.606643 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:06.607088 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:06.607328 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:06.607343 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.608424 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.608428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:06.609349 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:06.609361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.610034 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.610038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:06.610173 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:06.614717 (+ 4544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:06.614743 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:06.618293 (+ 3550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":377,"threads_started":1}
W20260808 02:45:06.618691 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:06.605600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:60101 (local address 127.25.137.254:41401)
0808 02:45:06.605770 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:06.605776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:06.606196 (+ 420us) server_negotiation.cc:407] Connection header received
0808 02:45:06.606388 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:06.606393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:06.606480 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:06.606592 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:06.607475 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.608289 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.609492 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.609916 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.610007 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:06.618494 (+ 8487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:60101: BlockingRecv error: recv got EOF from 127.25.137.195:60101 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 02:45:06 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:06 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:06.678821 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:06.669918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:51147)
0808 02:45:06.670510 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:06.670529 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:06.670662 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:06.671054 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:06.671060 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:06.671483 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:06.671658 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:06.671667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.672700 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.672705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:06.673686 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:06.673699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.674445 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.674451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:06.674618 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:06.675288 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:06.675308 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:06.678423 (+ 3115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":357,"threads_started":1}
W20260808 02:45:06.678887 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:06.669983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:51147 (local address 127.25.137.254:41401)
0808 02:45:06.670138 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:06.670143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:06.670640 (+ 497us) server_negotiation.cc:407] Connection header received
0808 02:45:06.670761 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:06.670765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:06.670881 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:06.670965 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:06.671778 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.672546 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.673816 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:06.674278 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:06.674375 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:06.678636 (+ 4261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:51147: BlockingRecv error: recv got EOF from 127.25.137.193:51147 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 02:45:07 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:07 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:07.144790 8102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:07.136614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:53399)
0808 02:45:07.137187 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:07.137203 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:07.137331 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:07.137617 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:07.137620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:07.138061 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:07.138219 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:07.138228 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.139195 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.139199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:07.140241 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:07.140258 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.140950 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.140955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:07.141125 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:07.141689 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:07.141708 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:07.144463 (+ 2755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":331,"threads_started":1}
W20260808 02:45:07.144927 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:07.136699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:53399 (local address 127.25.137.254:41401)
0808 02:45:07.136869 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:07.136874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:07.137268 (+ 394us) server_negotiation.cc:407] Connection header received
0808 02:45:07.137447 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:07.137450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:07.137513 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:07.137586 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:07.138326 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.139060 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.140378 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.140818 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.140937 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:07.144712 (+ 3775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:53399: BlockingRecv error: recv got EOF from 127.25.137.194:53399 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 02:45:07 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:07 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:07.630761 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:07.619606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:55009)
0808 02:45:07.620095 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:07.620110 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:07.620245 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:07.620603 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:07.620608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:07.621056 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:07.621258 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:07.621270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.622851 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.622857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:07.624327 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:07.624343 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.625476 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.625482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:07.625666 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:07.626764 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:07.626788 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:07.630437 (+ 3649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":280,"threads_started":1}
W20260808 02:45:07.630916 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:07.619639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:55009 (local address 127.25.137.254:41401)
0808 02:45:07.619826 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:07.619834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:07.620193 (+ 359us) server_negotiation.cc:407] Connection header received
0808 02:45:07.620368 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:07.620373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:07.620453 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:07.620543 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:07.621453 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.622605 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.624486 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.625110 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.625199 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:07.630667 (+ 5468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:55009: BlockingRecv error: recv got EOF from 127.25.137.195:55009 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 02:45:07 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:07 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:07.689196 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:07.679864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:34399)
0808 02:45:07.680483 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:07.680504 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:07.680642 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:07.681078 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:07.681083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:07.681533 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:07.681724 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:07.681745 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.682827 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.682831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:07.683837 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:07.683851 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.684678 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.684686 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:07.684923 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:07.685815 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:07.685840 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:07.688825 (+ 2985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":285,"threads_started":1}
W20260808 02:45:07.689289 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:07.679959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:34399 (local address 127.25.137.254:41401)
0808 02:45:07.680184 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:07.680192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:07.680614 (+ 422us) server_negotiation.cc:407] Connection header received
0808 02:45:07.680749 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:07.680756 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:07.680852 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:07.680960 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:07.681899 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.682661 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.684003 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:07.684480 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:07.684605 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:07.689035 (+ 4430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:34399: BlockingRecv error: recv got EOF from 127.25.137.193:34399 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 02:45:08 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:08 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:08.154197 8105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:08.145720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:53791)
0808 02:45:08.146234 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:08.146252 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:08.146376 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:08.146685 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:08.146689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:08.147108 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:08.147271 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:08.147281 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.148351 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.148356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:08.149390 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:08.149405 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.150085 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.150090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:08.150209 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:08.150781 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:08.150799 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:08.153944 (+ 3145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":255,"threads_started":1}
W20260808 02:45:08.154434 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:08.145941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:53791 (local address 127.25.137.254:41401)
0808 02:45:08.146168 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:08.146175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:08.146335 (+ 160us) server_negotiation.cc:407] Connection header received
0808 02:45:08.146467 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:08.146470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:08.146550 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:08.146631 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:08.147440 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.148208 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.149528 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.149925 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.150013 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:08.154150 (+ 4137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:53791: BlockingRecv error: recv got EOF from 127.25.137.194:53791 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 02:45:08 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:08 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:08.639838 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:08.631815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:34563)
0808 02:45:08.632415 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:08.632430 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:08.632543 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:08.632803 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:08.632806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:08.633225 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:08.633410 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:08.633421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.634495 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.634499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:08.635477 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:08.635491 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.636163 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.636167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:08.636279 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:08.636866 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:08.636884 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:08.639572 (+ 2688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":307,"threads_started":1}
W20260808 02:45:08.640050 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:08.631914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:34563 (local address 127.25.137.254:41401)
0808 02:45:08.632075 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:08.632079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:08.632523 (+ 444us) server_negotiation.cc:407] Connection header received
0808 02:45:08.632627 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:08.632630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:08.632695 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:08.632777 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:08.633568 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.634362 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.635612 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.636026 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.636117 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:08.639837 (+ 3720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:34563: BlockingRecv error: recv got EOF from 127.25.137.195:34563 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 02:45:08 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:08 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:08.698764 8107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:08.690149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:37875)
0808 02:45:08.690659 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:08.690675 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:08.690860 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:08.691222 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:08.691227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:08.691651 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:08.691846 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:08.691856 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.692887 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.692891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:08.693865 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:08.693880 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.694602 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.694608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:08.694787 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:08.695399 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:08.695417 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:08.698488 (+ 3071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":276,"threads_started":1}
W20260808 02:45:08.698920 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:08.690276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:37875 (local address 127.25.137.254:41401)
0808 02:45:08.690543 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:08.690549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:08.690760 (+ 211us) server_negotiation.cc:407] Connection header received
0808 02:45:08.690980 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:08.690985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:08.691086 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:08.691168 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:08.691989 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.692745 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.694003 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:08.694459 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:08.694568 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:08.698696 (+ 4128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:37875: BlockingRecv error: recv got EOF from 127.25.137.193:37875 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
Aug 08 02:45:09 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:09 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:09.163252 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:09.155180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:52595)
0808 02:45:09.155675 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:09.155694 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:09.155814 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:09.156075 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:09.156078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:09.156466 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:09.156646 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:09.156656 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.157844 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.157850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:09.158763 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:09.158776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.159466 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.159472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:09.159596 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:09.160201 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:09.160219 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:09.162990 (+ 2771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":255,"threads_started":1}
W20260808 02:45:09.163409 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:09.155278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:52595 (local address 127.25.137.254:41401)
0808 02:45:09.155468 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:09.155475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:09.155769 (+ 294us) server_negotiation.cc:407] Connection header received
0808 02:45:09.155898 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:09.155902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:09.155966 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:09.156047 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:09.156851 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.157690 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.158917 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.159341 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.159441 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:09.163190 (+ 3749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:52595: BlockingRecv error: recv got EOF from 127.25.137.194:52595 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 02:45:09 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:09 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.195@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:09.649559 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:09.640825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.195:39597)
0808 02:45:09.641355 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:09.641371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:09.641506 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:09.641822 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:09.641827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:09.642258 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:09.642513 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:09.642528 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.643720 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.643726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:09.644790 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:09.644805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.645543 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.645548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:09.645681 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:09.646265 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:09.646284 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:09.649268 (+ 2984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":276,"threads_started":1}
W20260808 02:45:09.649637 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:09.640963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:39597 (local address 127.25.137.254:41401)
0808 02:45:09.641198 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:09.641205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:09.641449 (+ 244us) server_negotiation.cc:407] Connection header received
0808 02:45:09.641627 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:09.641631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:09.641695 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:09.641774 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:09.642688 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.643556 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.644924 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.645391 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.645485 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:09.649457 (+ 3972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.195:39597: BlockingRecv error: recv got EOF from 127.25.137.195:39597 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 02:45:09 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:09 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.193@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:09.708519 8113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:09.700059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.193:59791)
0808 02:45:09.700614 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:09.700630 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:09.700764 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:09.701141 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:09.701145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:09.701562 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:09.701733 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:09.701742 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.702807 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.702811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:09.703868 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:09.703885 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.704652 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.704658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:09.704822 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:09.705433 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:09.705450 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:09.708158 (+ 2708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":312,"threads_started":1}
W20260808 02:45:09.708617 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:09.700147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:59791 (local address 127.25.137.254:41401)
0808 02:45:09.700334 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:09.700341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:09.700714 (+ 373us) server_negotiation.cc:407] Connection header received
0808 02:45:09.700880 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:09.700884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:09.700966 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:09.701089 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:09.701914 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.702654 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.704007 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:09.704506 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:09.704619 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:09.708385 (+ 3766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.193:59791: BlockingRecv error: recv got EOF from 127.25.137.193:59791 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 02:45:10 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:45:10 dist-test-slave-k5rr krb5kdc[6503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.137.194@KRBTEST.COM for oryx/127.25.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:45:10.173502 8114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:10.164508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41401 (local address 127.25.137.194:55949)
0808 02:45:10.165120 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:10.165136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:10.165252 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:10.165576 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:10.165580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:10.166033 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:10.166207 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:10.166217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:10.167368 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:10.167373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:10.168429 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:10.168444 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:10.169377 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:10.169382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:10.169555 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:10.170213 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:10.170253 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:10.173229 (+ 2976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.137.254:41401: Server oryx/127.25.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":315,"threads_started":1}
W20260808 02:45:10.173679 7633 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:10.164607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:55949 (local address 127.25.137.254:41401)
0808 02:45:10.164766 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:10.164771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:10.165252 (+ 481us) server_negotiation.cc:407] Connection header received
0808 02:45:10.165364 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:10.165369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:10.165438 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:10.165522 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:10.166372 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:10.167192 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:10.168598 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:10.169221 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:10.169330 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:10.173458 (+ 4128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.137.194:55949: BlockingRecv error: recv got EOF from 127.25.137.194:55949 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:45:10.468740 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 7414
I20260808 02:45:10.481210 8118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:10.469949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49046 (local address 127.25.137.193:38765)
0808 02:45:10.470522 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:10.470530 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:10.470557 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:45:10.470615 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:10.470622 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:10.470854 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:10.471021 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:10.472480 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:10.473581 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:10.474909 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:10.475461 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:10.475558 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:10.478104 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:10.478112 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:10.478125 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:45:10.478159 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:10.480695 (+ 2536us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.137.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 02:45:10.480954 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49046: Request ticket server kudu/127.25.137.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":309,"threads_started":1}
W20260808 02:45:10.481324 8118 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49046: Request ticket server kudu/127.25.137.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 02:45:10.481434 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 7414: Not authorized: Client connection negotiation failed: client connection to 127.25.137.193:38765: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.137.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 02:45:10.481632 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 7414
I20260808 02:45:10.506888 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 7635
I20260808 02:45:10.519276 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:10.507950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47622 (local address 127.25.137.194:36359)
0808 02:45:10.508577 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:10.508585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:10.508617 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:45:10.508681 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:10.508688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:10.508916 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:10.509125 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:10.510476 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:10.511584 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:10.512821 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:10.513425 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:10.513544 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:10.515959 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:10.515968 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:10.515983 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:45:10.516022 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:10.518621 (+ 2599us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.137.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 02:45:10.518940 (+ 319us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47622: Request ticket server kudu/127.25.137.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":317,"threads_started":1}
W20260808 02:45:10.519361 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 7635: Not authorized: Client connection negotiation failed: client connection to 127.25.137.194:36359: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.137.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 02:45:10.519400 8119 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47622: Request ticket server kudu/127.25.137.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 02:45:10.519532 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 7635
I20260808 02:45:10.547003 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 7845
I20260808 02:45:10.559909 8120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:10.548229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33918 (local address 127.25.137.195:36895)
0808 02:45:10.548849 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:10.548856 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:10.548884 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:45:10.548947 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:10.548954 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:10.549221 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:10.549384 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:10.550874 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:10.552009 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:10.553431 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:10.554022 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:10.554124 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:10.556760 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:10.556769 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:10.556796 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:45:10.556832 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:10.559383 (+ 2551us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.137.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 02:45:10.559644 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33918: Request ticket server kudu/127.25.137.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":312,"threads_started":1}
W20260808 02:45:10.560025 8120 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33918: Request ticket server kudu/127.25.137.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 02:45:10.560161 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 7845: Not authorized: Client connection negotiation failed: client connection to 127.25.137.195:36895: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.137.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 02:45:10.560351 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 7845
I20260808 02:45:10.584597 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 7307
I20260808 02:45:10.586031 7404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:10.706925 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 7307
2026-08-08T02:45:10Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21394 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T02:45:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:45:10Z Disabled control of system clock
I20260808 02:45:10.800124 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45457
--webserver_interface=127.25.137.254
--webserver_port=0
--builtin_ntp_servers=127.25.137.212:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:45457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 02:45:11.131320 8130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:11.131631 8130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:11.131754 8130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:11.138952 8130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:45:11.139055 8130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:11.139117 8130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:11.139189 8130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:45:11.139248 8130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:45:11.148389 8130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:45457
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:11.149736 8130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:11.151574 8130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:11.162825 8138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:11.162990 8140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:11.162990 8142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:11.163924 8130 server_base.cc:1034] running on GCE node
I20260808 02:45:11.164616 8130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:11.166038 8130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:11.167239 8130 hybrid_clock.cc:648] HybridClock initialized: now 1786157111167214 us; error 52 us; skew 500 ppm
I20260808 02:45:11.167553 8130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:45:11.170178 8130 webserver.cc:460] Webserver started at http://127.25.137.254:40927/ using document root <none> and password file <none>
I20260808 02:45:11.170876 8130 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:11.170972 8130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:11.171253 8130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:11.174224 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "713d8b970bea44489c0e130fbb51320e"
format_stamp: "Formatted at 2026-08-08 02:45:11 on dist-test-slave-k5rr"
server_key: "f60edc826ca16c579b3b4c58b119260b"
server_key_iv: "dc46c34d5d6afa0b3060f0431a28d56f"
server_key_version: "encryptionkey@0"
I20260808 02:45:11.174978 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "713d8b970bea44489c0e130fbb51320e"
format_stamp: "Formatted at 2026-08-08 02:45:11 on dist-test-slave-k5rr"
server_key: "f60edc826ca16c579b3b4c58b119260b"
server_key_iv: "dc46c34d5d6afa0b3060f0431a28d56f"
server_key_version: "encryptionkey@0"
I20260808 02:45:11.179973 8130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260808 02:45:11.183238 8155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:11.184473 8130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:45:11.184633 8130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "713d8b970bea44489c0e130fbb51320e"
format_stamp: "Formatted at 2026-08-08 02:45:11 on dist-test-slave-k5rr"
server_key: "f60edc826ca16c579b3b4c58b119260b"
server_key_iv: "dc46c34d5d6afa0b3060f0431a28d56f"
server_key_version: "encryptionkey@0"
I20260808 02:45:11.184835 8130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:11.199958 8130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:11.200768 8130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:11.201089 8130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:11.217855 8130 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:45457
I20260808 02:45:11.217890 8236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:45457 every 8 connection(s)
I20260808 02:45:11.219146 8130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:45:11.222779 8237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:45:11.224900 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 8130
I20260808 02:45:11.225102 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:45:11.225754 26151 external_mini_cluster.cc:1428] Setting key dc24f6a8468b467db11166729b330c21
I20260808 02:45:11.233942 8237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e: Bootstrap starting.
I20260808 02:45:11.237160 8237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e: Neither blocks nor log segments found. Creating new log.
I20260808 02:45:11.238262 8237 log.cc:824] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:45:11.239214 8244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:11.227841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.25.137.254:45457)
0808 02:45:11.228786 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:11.228797 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:11.228840 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:45:11.230518 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:11.230540 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:11.231003 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:11.231447 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:11.232061 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:11.233361 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:11.234676 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:11.235388 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:11.235553 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:11.236714 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:11.236722 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:11.236735 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:11.237360 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:11.237456 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:11.237485 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:11.237602 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:11.237802 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 02:45:11.238105 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":325,"threads_started":1}
I20260808 02:45:11.241487 8237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e: No bootstrap required, opened a new log
I20260808 02:45:11.248957 8237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713d8b970bea44489c0e130fbb51320e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45457 } }
I20260808 02:45:11.249346 8237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:45:11.249437 8237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713d8b970bea44489c0e130fbb51320e, State: Initialized, Role: FOLLOWER
I20260808 02:45:11.249976 8237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713d8b970bea44489c0e130fbb51320e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45457 } }
I20260808 02:45:11.250196 8237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:45:11.250319 8237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:45:11.250473 8237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:45:11.252616 8237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713d8b970bea44489c0e130fbb51320e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45457 } }
I20260808 02:45:11.253171 8237 leader_election.cc:302] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 713d8b970bea44489c0e130fbb51320e; no voters:
I20260808 02:45:11.253923 8237 leader_election.cc:288] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:45:11.254047 8246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:45:11.254721 8246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [term 1 LEADER]: Becoming Leader. State: Replica: 713d8b970bea44489c0e130fbb51320e, State: Running, Role: LEADER
I20260808 02:45:11.255276 8246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713d8b970bea44489c0e130fbb51320e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45457 } }
I20260808 02:45:11.256004 8237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:45:11.258451 8252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 713d8b970bea44489c0e130fbb51320e. Latest consensus state: current_term: 1 leader_uuid: "713d8b970bea44489c0e130fbb51320e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713d8b970bea44489c0e130fbb51320e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45457 } } }
I20260808 02:45:11.259109 8252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [sys.catalog]: This master's current role is: LEADER
I20260808 02:45:11.258668 8251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "713d8b970bea44489c0e130fbb51320e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713d8b970bea44489c0e130fbb51320e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45457 } } }
I20260808 02:45:11.259606 8251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e [sys.catalog]: This master's current role is: LEADER
I20260808 02:45:11.261377 8268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:45:11.266577 8268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:45:11.275098 8268 catalog_manager.cc:1347] Generated new cluster ID: a69a413cf3a448209753ccaadb4084b3
I20260808 02:45:11.275210 8268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:45:11.301932 8268 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:45:11.303290 8268 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:45:11.314744 8268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 713d8b970bea44489c0e130fbb51320e: Generated new TSK 0
I20260808 02:45:11.315582 8268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:45:11.327545 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45457
--builtin_ntp_servers=127.25.137.212:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 02:45:11.687364 8282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:11.687633 8282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:11.687716 8282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:11.695117 8282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:11.695215 8282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:11.695322 8282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:45:11.704396 8282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:11.705705 8282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:11.707509 8282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:11.720609 8289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:11.720702 8290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:11.720676 8294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:11.721617 8282 server_base.cc:1034] running on GCE node
I20260808 02:45:11.722218 8282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:11.723160 8282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:11.724373 8282 hybrid_clock.cc:648] HybridClock initialized: now 1786157111724310 us; error 54 us; skew 500 ppm
I20260808 02:45:11.724694 8282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:45:11.730371 8282 webserver.cc:460] Webserver started at http://127.25.137.193:35991/ using document root <none> and password file <none>
I20260808 02:45:11.731096 8282 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:11.731202 8282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:11.731544 8282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:11.734486 8282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "7816c9869b064adf89bc5e23052b871f"
format_stamp: "Formatted at 2026-08-08 02:45:11 on dist-test-slave-k5rr"
server_key: "335c1ff8a8812c2abfc2649352502508"
server_key_iv: "1bfd4a38bbf1f78ff2326a7f91f46193"
server_key_version: "encryptionkey@0"
I20260808 02:45:11.735296 8282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "7816c9869b064adf89bc5e23052b871f"
format_stamp: "Formatted at 2026-08-08 02:45:11 on dist-test-slave-k5rr"
server_key: "335c1ff8a8812c2abfc2649352502508"
server_key_iv: "1bfd4a38bbf1f78ff2326a7f91f46193"
server_key_version: "encryptionkey@0"
I20260808 02:45:11.740595 8282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260808 02:45:11.744046 8302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:11.745424 8282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:45:11.745571 8282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "7816c9869b064adf89bc5e23052b871f"
format_stamp: "Formatted at 2026-08-08 02:45:11 on dist-test-slave-k5rr"
server_key: "335c1ff8a8812c2abfc2649352502508"
server_key_iv: "1bfd4a38bbf1f78ff2326a7f91f46193"
server_key_version: "encryptionkey@0"
I20260808 02:45:11.745769 8282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:11.797603 8282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:11.798525 8282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:11.798831 8282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:11.799829 8282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:45:11.801405 8282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:45:11.801505 8282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:11.801614 8282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:45:11.801687 8282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:11.832135 8282 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:44975
I20260808 02:45:11.832193 8496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:44975 every 8 connection(s)
I20260808 02:45:11.833521 8282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:45:11.841255 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 8282
I20260808 02:45:11.841449 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:45:11.842280 26151 external_mini_cluster.cc:1428] Setting key 197635d282ab060095e84eb9787a0f22
I20260808 02:45:11.845052 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45457
--builtin_ntp_servers=127.25.137.212:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 02:45:11.846210 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:11.835964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:37739 (local address 127.25.137.254:45457)
0808 02:45:11.836409 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:11.836413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:11.837523 (+ 1110us) server_negotiation.cc:407] Connection header received
0808 02:45:11.839385 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:11.839391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:11.839472 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:11.839569 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:11.840634 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:11.841436 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:11.843488 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:11.843921 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:11.844036 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:11.845385 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:11.845391 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:11.845394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:11.845412 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:11.845458 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:11.845471 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:11.845546 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:11.845763 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 02:45:11.845868 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":235,"threads_started":1}
I20260808 02:45:11.847167 8501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:11.836454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45457 (local address 127.25.137.193:37739)
0808 02:45:11.837365 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:11.837408 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:11.839106 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:11.839811 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:11.839819 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:11.839903 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:11.840464 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:11.840481 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:11.841639 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:11.841646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:11.843323 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:11.843345 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:11.844153 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:11.844160 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:11.844377 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:11.845071 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:45:11.845073 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:11.845111 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:45:11.845204 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:45:11.845560 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:45:11.845572 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:45:11.845580 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:45:11.845740 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 02:45:11.846124 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":274,"threads_started":1}
I20260808 02:45:11.848650 8497 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45457
I20260808 02:45:11.848961 8497 heartbeater.cc:464] Registering TS with master...
I20260808 02:45:11.849871 8497 heartbeater.cc:511] Master 127.25.137.254:45457 requested a full tablet report, sending...
I20260808 02:45:11.852149 8182 ts_manager.cc:194] Registered new tserver with Master: 7816c9869b064adf89bc5e23052b871f (127.25.137.193:44975)
I20260808 02:45:11.854035 8182 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.193:37739
W20260808 02:45:12.181027 8502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:12.181349 8502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:12.181474 8502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:12.188917 8502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:12.189085 8502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:12.189260 8502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:45:12.198848 8502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:12.200209 8502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:12.202045 8502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:12.215060 8508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:12.215083 8511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:12.215546 8509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:12.215885 8502 server_base.cc:1034] running on GCE node
I20260808 02:45:12.216490 8502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:12.217583 8502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:12.218824 8502 hybrid_clock.cc:648] HybridClock initialized: now 1786157112218762 us; error 52 us; skew 500 ppm
I20260808 02:45:12.219195 8502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:45:12.221896 8502 webserver.cc:460] Webserver started at http://127.25.137.194:33689/ using document root <none> and password file <none>
I20260808 02:45:12.222796 8502 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:12.222924 8502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:12.223291 8502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:12.227121 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "82286848c93643a985c407512f4e5426"
format_stamp: "Formatted at 2026-08-08 02:45:12 on dist-test-slave-k5rr"
server_key: "43d90444ab9035d886ac9744669ac16b"
server_key_iv: "44b44a99679d49d612230885108650e5"
server_key_version: "encryptionkey@0"
I20260808 02:45:12.228092 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "82286848c93643a985c407512f4e5426"
format_stamp: "Formatted at 2026-08-08 02:45:12 on dist-test-slave-k5rr"
server_key: "43d90444ab9035d886ac9744669ac16b"
server_key_iv: "44b44a99679d49d612230885108650e5"
server_key_version: "encryptionkey@0"
I20260808 02:45:12.234813 8502 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260808 02:45:12.239224 8523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:12.240716 8502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:45:12.240906 8502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "82286848c93643a985c407512f4e5426"
format_stamp: "Formatted at 2026-08-08 02:45:12 on dist-test-slave-k5rr"
server_key: "43d90444ab9035d886ac9744669ac16b"
server_key_iv: "44b44a99679d49d612230885108650e5"
server_key_version: "encryptionkey@0"
I20260808 02:45:12.241168 8502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:12.266213 8502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:12.267127 8502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:12.267431 8502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:12.268488 8502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:45:12.270046 8502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:45:12.270145 8502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:12.270254 8502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:45:12.270326 8502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:12.301352 8502 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:44357
I20260808 02:45:12.301445 8700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:44357 every 8 connection(s)
I20260808 02:45:12.302652 8502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:45:12.304852 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 8502
I20260808 02:45:12.305042 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:45:12.305537 26151 external_mini_cluster.cc:1428] Setting key 69f32e6e81ba1ff2ac86bd6e4cb0eb41
I20260808 02:45:12.308749 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45457
--builtin_ntp_servers=127.25.137.212:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 02:45:12.316537 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:12.305416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:34177 (local address 127.25.137.254:45457)
0808 02:45:12.305628 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:12.305634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:12.307145 (+ 1511us) server_negotiation.cc:407] Connection header received
0808 02:45:12.309562 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:12.309566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:12.309632 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:12.309718 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:12.311064 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.311792 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.313821 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.314214 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.314310 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:12.315774 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:12.315799 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:12.315802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:12.315820 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:12.315854 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:12.315866 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:12.315935 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:12.316224 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 02:45:12.316303 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:45:12.317960 8704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:12.305952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45457 (local address 127.25.137.194:34177)
0808 02:45:12.306992 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:12.307029 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:12.309268 (+ 2239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:12.309978 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:12.309989 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:12.310113 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:12.310911 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:12.310936 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.311955 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.311960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:12.313679 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:12.313694 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.314437 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.314443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:12.314706 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:12.315438 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:45:12.315441 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:12.315491 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:45:12.315592 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:45:12.315958 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:45:12.315969 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:45:12.315976 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:45:12.316214 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 02:45:12.316617 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":353,"threads_started":1}
I20260808 02:45:12.319521 8702 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45457
I20260808 02:45:12.319849 8702 heartbeater.cc:464] Registering TS with master...
I20260808 02:45:12.320761 8702 heartbeater.cc:511] Master 127.25.137.254:45457 requested a full tablet report, sending...
I20260808 02:45:12.322455 8182 ts_manager.cc:194] Registered new tserver with Master: 82286848c93643a985c407512f4e5426 (127.25.137.194:44357)
I20260808 02:45:12.323606 8182 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.194:34177
W20260808 02:45:12.653812 8705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:12.654110 8705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:12.654213 8705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:12.661614 8705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:12.661727 8705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:12.661859 8705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:45:12.671159 8705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:12.672437 8705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:12.674336 8705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:12.687557 8716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:12.687597 8718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:12.687875 8715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:12.688678 8705 server_base.cc:1034] running on GCE node
I20260808 02:45:12.689388 8705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:12.690366 8705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:12.691583 8705 hybrid_clock.cc:648] HybridClock initialized: now 1786157112691530 us; error 41 us; skew 500 ppm
I20260808 02:45:12.691962 8705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:45:12.694236 8705 webserver.cc:460] Webserver started at http://127.25.137.195:44587/ using document root <none> and password file <none>
I20260808 02:45:12.694941 8705 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:12.695034 8705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:12.695353 8705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:12.698382 8705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "bf75e54252f34ce78d9b032d560451e1"
format_stamp: "Formatted at 2026-08-08 02:45:12 on dist-test-slave-k5rr"
server_key: "ef7b64e7be64100dbc665c698fa3cd63"
server_key_iv: "7e0d834c12d59e1c7a4909240d3ef53e"
server_key_version: "encryptionkey@0"
I20260808 02:45:12.699195 8705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "bf75e54252f34ce78d9b032d560451e1"
format_stamp: "Formatted at 2026-08-08 02:45:12 on dist-test-slave-k5rr"
server_key: "ef7b64e7be64100dbc665c698fa3cd63"
server_key_iv: "7e0d834c12d59e1c7a4909240d3ef53e"
server_key_version: "encryptionkey@0"
I20260808 02:45:12.704532 8705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260808 02:45:12.708230 8728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:12.709600 8705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:45:12.709745 8705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "bf75e54252f34ce78d9b032d560451e1"
format_stamp: "Formatted at 2026-08-08 02:45:12 on dist-test-slave-k5rr"
server_key: "ef7b64e7be64100dbc665c698fa3cd63"
server_key_iv: "7e0d834c12d59e1c7a4909240d3ef53e"
server_key_version: "encryptionkey@0"
I20260808 02:45:12.709957 8705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:12.723467 8705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:12.724291 8705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:12.724574 8705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:12.725594 8705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:45:12.727010 8705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:45:12.727097 8705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:12.727202 8705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:45:12.727268 8705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:12.758834 8705 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:36469
I20260808 02:45:12.758869 8893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:36469 every 8 connection(s)
I20260808 02:45:12.760141 8705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:45:12.768920 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 8705
I20260808 02:45:12.769126 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:45:12.769659 26151 external_mini_cluster.cc:1428] Setting key c5514ecd944e3a27964c7643a589e749
I20260808 02:45:12.773113 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:12.762720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:54691 (local address 127.25.137.254:45457)
0808 02:45:12.762945 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:12.762950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:12.764264 (+ 1314us) server_negotiation.cc:407] Connection header received
0808 02:45:12.766100 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:12.766105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:12.766191 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:12.766282 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:12.767441 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.768292 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.770208 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.770675 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.770814 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:12.772223 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:12.772228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:12.772231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:12.772248 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:12.772280 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:12.772315 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:12.772402 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:12.772749 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 02:45:12.772831 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260808 02:45:12.773967 8902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:12.763196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45457 (local address 127.25.137.195:54691)
0808 02:45:12.764110 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:12.764145 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:12.765832 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:12.766519 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:12.766528 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:12.766610 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:12.767154 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:12.767171 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.768472 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.768476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:12.770038 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:12.770053 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.770909 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.770918 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:12.771156 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:12.771886 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:45:12.771890 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:12.771942 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:45:12.772038 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:45:12.772421 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:45:12.772433 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:45:12.772439 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:45:12.772585 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 02:45:12.772899 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":267,"threads_started":1}
I20260808 02:45:12.775259 8896 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45457
I20260808 02:45:12.775532 8896 heartbeater.cc:464] Registering TS with master...
I20260808 02:45:12.776237 8896 heartbeater.cc:511] Master 127.25.137.254:45457 requested a full tablet report, sending...
I20260808 02:45:12.777561 8182 ts_manager.cc:194] Registered new tserver with Master: bf75e54252f34ce78d9b032d560451e1 (127.25.137.195:36469)
I20260808 02:45:12.778318 8182 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.195:54691
I20260808 02:45:12.784782 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:45:12.791072 8500 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:12.789425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55940 (local address 127.25.137.254:45457)
0808 02:45:12.789589 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:12.789594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:12.790029 (+ 435us) server_negotiation.cc:407] Connection header received
0808 02:45:12.790233 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:12.790237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:12.790304 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:12.790388 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:12.790823 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55940: BlockingRecv error: recv got EOF from 127.0.0.1:55940 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:45:12.791975 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 8282
I20260808 02:45:12.799949 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:12.792878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45548 (local address 127.25.137.193:44975)
0808 02:45:12.793856 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:12.793862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:12.793887 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:45:12.793941 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:12.793946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:12.794142 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:12.794257 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:12.794655 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.795642 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.796798 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.797416 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.797507 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:12.798480 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:12.798488 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:12.798500 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:12.799101 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:12.799148 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:12.799167 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:12.799251 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:12.799486 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 02:45:12.799597 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":837,"thread_start_us":729,"threads_started":1}
I20260808 02:45:12.801218 8490 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:12.881203 8497 heartbeater.cc:503] Master 127.25.137.254:45457 was elected leader, sending a full tablet report...
I20260808 02:45:12.883265 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 8282
I20260808 02:45:12.903025 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 8502
I20260808 02:45:12.911442 8912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:12.903980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.25.137.194:44357)
0808 02:45:12.904520 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:12.904527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:12.904557 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:45:12.904631 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:12.904638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:12.904856 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:12.905037 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:12.905494 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.906529 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.907881 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:12.908502 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:12.908642 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:12.909896 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:12.909904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:12.909917 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:12.910539 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:12.910592 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:12.910613 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:12.910726 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:12.910918 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 02:45:12.911062 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":266,"threads_started":1}
I20260808 02:45:12.912726 8691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:12.993896 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 8502
I20260808 02:45:13.025516 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 8705
I20260808 02:45:13.036242 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:13.026763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51436 (local address 127.25.137.195:36469)
0808 02:45:13.027430 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:13.027438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:13.027620 (+ 182us) server_negotiation.cc:407] Connection header received
0808 02:45:13.027832 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:13.027840 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:13.028050 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:13.028179 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:13.028731 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:13.030417 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:13.032608 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:13.033294 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:13.033416 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:13.034604 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:13.034617 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:13.034635 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:13.035380 (+ 745us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:13.035441 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:13.035470 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:13.035560 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:13.035737 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 02:45:13.035865 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":338,"threads_started":1}
I20260808 02:45:13.037758 8886 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:13.130357 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 8705
I20260808 02:45:13.155328 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 8130
I20260808 02:45:13.156574 8230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:13.259398 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 8130
2026-08-08T02:45:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2550 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T02:45:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:45:13Z Disabled control of system clock
I20260808 02:45:13.350908 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:41781
--webserver_interface=127.25.137.254
--webserver_port=0
--builtin_ntp_servers=127.25.137.212:45813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:41781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:45:13.692260 8929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:13.692570 8929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:13.692669 8929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:13.700263 8929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:45:13.700398 8929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:13.700449 8929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:13.700521 8929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:45:13.700573 8929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:45:13.710314 8929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:41781
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:41781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:13.711670 8929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:13.713603 8929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:13.725492 8936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:13.725533 8940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:13.725553 8938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:13.726403 8929 server_base.cc:1034] running on GCE node
I20260808 02:45:13.727226 8929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:13.728638 8929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:13.729885 8929 hybrid_clock.cc:648] HybridClock initialized: now 1786157113729816 us; error 51 us; skew 500 ppm
I20260808 02:45:13.730232 8929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:45:13.732579 8929 webserver.cc:460] Webserver started at http://127.25.137.254:35615/ using document root <none> and password file <none>
I20260808 02:45:13.733323 8929 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:13.733433 8929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:13.733741 8929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:13.736590 8929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "e6d63cb38ecc46d3bfaf102000d66a5a"
format_stamp: "Formatted at 2026-08-08 02:45:13 on dist-test-slave-k5rr"
server_key: "872c6678f87fedf088b87579517d25a1"
server_key_iv: "8a0568e3fc538a0ad8302857fe6727ed"
server_key_version: "encryptionkey@0"
I20260808 02:45:13.737438 8929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "e6d63cb38ecc46d3bfaf102000d66a5a"
format_stamp: "Formatted at 2026-08-08 02:45:13 on dist-test-slave-k5rr"
server_key: "872c6678f87fedf088b87579517d25a1"
server_key_iv: "8a0568e3fc538a0ad8302857fe6727ed"
server_key_version: "encryptionkey@0"
I20260808 02:45:13.742576 8929 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:45:13.745940 8949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:13.747241 8929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:45:13.747401 8929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "e6d63cb38ecc46d3bfaf102000d66a5a"
format_stamp: "Formatted at 2026-08-08 02:45:13 on dist-test-slave-k5rr"
server_key: "872c6678f87fedf088b87579517d25a1"
server_key_iv: "8a0568e3fc538a0ad8302857fe6727ed"
server_key_version: "encryptionkey@0"
I20260808 02:45:13.747597 8929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:13.753332 8929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:13.754065 8929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:13.754336 8929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:13.771837 8929 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:41781
I20260808 02:45:13.771857 9031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:41781 every 8 connection(s)
I20260808 02:45:13.773216 8929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:45:13.776139 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 8929
I20260808 02:45:13.776314 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:45:13.776890 9032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:45:13.776997 26151 external_mini_cluster.cc:1428] Setting key ad064c52d255c7daa2925f537b570f8b
I20260808 02:45:13.785044 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:13.778936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54882 (local address 127.25.137.254:41781)
0808 02:45:13.779942 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:13.779952 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:13.779995 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:45:13.781661 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:13.781685 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:13.782090 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:13.782404 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:13.783464 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:13.783470 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:13.783481 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:13.783499 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:13.783559 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:13.783574 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:13.783751 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:13.783904 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 02:45:13.784165 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":377,"threads_started":1}
I20260808 02:45:13.789502 9032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a: Bootstrap starting.
I20260808 02:45:13.792428 9032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a: Neither blocks nor log segments found. Creating new log.
I20260808 02:45:13.793421 9032 log.cc:824] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:45:13.796144 9032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a: No bootstrap required, opened a new log
I20260808 02:45:13.803483 9032 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d63cb38ecc46d3bfaf102000d66a5a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41781 } }
I20260808 02:45:13.803838 9032 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:45:13.803947 9032 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d63cb38ecc46d3bfaf102000d66a5a, State: Initialized, Role: FOLLOWER
I20260808 02:45:13.804478 9032 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d63cb38ecc46d3bfaf102000d66a5a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41781 } }
I20260808 02:45:13.804695 9032 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:45:13.804823 9032 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:45:13.804981 9032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:45:13.807109 9032 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d63cb38ecc46d3bfaf102000d66a5a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41781 } }
I20260808 02:45:13.807605 9032 leader_election.cc:302] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6d63cb38ecc46d3bfaf102000d66a5a; no voters:
I20260808 02:45:13.808169 9032 leader_election.cc:288] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:45:13.808296 9041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:45:13.809397 9041 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [term 1 LEADER]: Becoming Leader. State: Replica: e6d63cb38ecc46d3bfaf102000d66a5a, State: Running, Role: LEADER
I20260808 02:45:13.809929 9041 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d63cb38ecc46d3bfaf102000d66a5a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41781 } }
I20260808 02:45:13.810247 9032 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:45:13.812515 9045 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6d63cb38ecc46d3bfaf102000d66a5a. Latest consensus state: current_term: 1 leader_uuid: "e6d63cb38ecc46d3bfaf102000d66a5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d63cb38ecc46d3bfaf102000d66a5a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41781 } } }
I20260808 02:45:13.812930 9045 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [sys.catalog]: This master's current role is: LEADER
I20260808 02:45:13.813555 9042 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6d63cb38ecc46d3bfaf102000d66a5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d63cb38ecc46d3bfaf102000d66a5a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41781 } } }
I20260808 02:45:13.813905 9042 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a [sys.catalog]: This master's current role is: LEADER
I20260808 02:45:13.816700 9057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:45:13.820948 9057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:45:13.828563 9057 catalog_manager.cc:1347] Generated new cluster ID: bbc60d214d464519a267a3275a54e4c9
I20260808 02:45:13.828662 9057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:45:13.860642 9057 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:45:13.861076 9057 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:45:13.870393 9057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6d63cb38ecc46d3bfaf102000d66a5a: Generated new TSK 0
I20260808 02:45:13.871392 9057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:45:13.878621 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41781
--builtin_ntp_servers=127.25.137.212:45813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:45:14.211256 9075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:14.211568 9075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:14.211709 9075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:14.219386 9075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:14.219488 9075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:14.219624 9075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:45:14.228781 9075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:14.230119 9075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:14.231940 9075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:14.244544 9082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:14.244547 9084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:14.244750 9088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:14.245383 9075 server_base.cc:1034] running on GCE node
I20260808 02:45:14.245976 9075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:14.246873 9075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:14.248076 9075 hybrid_clock.cc:648] HybridClock initialized: now 1786157114248028 us; error 34 us; skew 500 ppm
I20260808 02:45:14.248368 9075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:45:14.250597 9075 webserver.cc:460] Webserver started at http://127.25.137.193:42499/ using document root <none> and password file <none>
I20260808 02:45:14.251282 9075 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:14.251389 9075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:14.251734 9075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:14.256660 9075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "2eeb6e3b16f44beba4a882c430c3bf59"
format_stamp: "Formatted at 2026-08-08 02:45:14 on dist-test-slave-k5rr"
server_key: "7925d2286f95580f13b57c0e77ec95a6"
server_key_iv: "aeb4710130aee72bd486cf5d626fbcf7"
server_key_version: "encryptionkey@0"
I20260808 02:45:14.257432 9075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "2eeb6e3b16f44beba4a882c430c3bf59"
format_stamp: "Formatted at 2026-08-08 02:45:14 on dist-test-slave-k5rr"
server_key: "7925d2286f95580f13b57c0e77ec95a6"
server_key_iv: "aeb4710130aee72bd486cf5d626fbcf7"
server_key_version: "encryptionkey@0"
I20260808 02:45:14.262462 9075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:45:14.265870 9097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:14.267274 9075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:45:14.267437 9075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "2eeb6e3b16f44beba4a882c430c3bf59"
format_stamp: "Formatted at 2026-08-08 02:45:14 on dist-test-slave-k5rr"
server_key: "7925d2286f95580f13b57c0e77ec95a6"
server_key_iv: "aeb4710130aee72bd486cf5d626fbcf7"
server_key_version: "encryptionkey@0"
I20260808 02:45:14.267642 9075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:14.273267 9075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:14.274041 9075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:14.274283 9075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:14.275172 9075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:45:14.276629 9075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:45:14.276726 9075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:14.276837 9075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:45:14.276913 9075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:14.307806 9075 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:41811
I20260808 02:45:14.307864 9272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:41811 every 8 connection(s)
I20260808 02:45:14.309152 9075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:45:14.317163 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 9075
I20260808 02:45:14.317358 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:45:14.317446 9275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:14.311655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:38003 (local address 127.25.137.254:41781)
0808 02:45:14.312126 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:14.312131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:14.313294 (+ 1163us) server_negotiation.cc:407] Connection header received
0808 02:45:14.315105 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:14.315111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:14.315181 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:14.315286 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:14.316558 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:14.316564 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:14.316570 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:14.316589 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:14.316641 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:14.316658 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:14.316780 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:14.317068 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 02:45:14.317138 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":269,"threads_started":1}
I20260808 02:45:14.318080 26151 external_mini_cluster.cc:1428] Setting key 530ff80245bf7225399f56245dc6bf8c
I20260808 02:45:14.318363 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:14.312175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41781 (local address 127.25.137.193:38003)
0808 02:45:14.313121 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:14.313157 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:14.314848 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:14.315548 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:14.315556 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:14.315643 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:14.316274 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:45:14.316277 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:14.316313 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:45:14.316392 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:45:14.316788 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:45:14.316801 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:45:14.316811 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 02:45:14.317120 (+ 309us) client_negotiation.cc:240] Negotiation successful
0808 02:45:14.317355 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":259,"threads_started":1}
I20260808 02:45:14.319799 9273 heartbeater.cc:347] Connected to a master server at 127.25.137.254:41781
I20260808 02:45:14.320075 9273 heartbeater.cc:464] Registering TS with master...
I20260808 02:45:14.320674 9273 heartbeater.cc:511] Master 127.25.137.254:41781 requested a full tablet report, sending...
I20260808 02:45:14.321049 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41781
--builtin_ntp_servers=127.25.137.212:45813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 02:45:14.322777 8979 ts_manager.cc:194] Registered new tserver with Master: 2eeb6e3b16f44beba4a882c430c3bf59 (127.25.137.193:41811)
W20260808 02:45:14.660852 9277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:14.661195 9277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:14.661314 9277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:14.669271 9277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:14.669391 9277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:14.669528 9277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:45:14.679016 9277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:14.680301 9277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:14.682233 9277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:14.695976 9283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:14.696017 9285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:14.695997 9287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:14.696813 9277 server_base.cc:1034] running on GCE node
I20260808 02:45:14.697553 9277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:14.698678 9277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:14.699918 9277 hybrid_clock.cc:648] HybridClock initialized: now 1786157114699853 us; error 54 us; skew 500 ppm
I20260808 02:45:14.700270 9277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:45:14.702796 9277 webserver.cc:460] Webserver started at http://127.25.137.194:41311/ using document root <none> and password file <none>
I20260808 02:45:14.703544 9277 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:14.703660 9277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:14.704002 9277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:14.707125 9277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "3e694296935a4daba8cf9be03f5034f1"
format_stamp: "Formatted at 2026-08-08 02:45:14 on dist-test-slave-k5rr"
server_key: "a96b5259aa7b7c9a8e8bc9a81ad295fd"
server_key_iv: "cb6ec8aad0b89158b9165618ccb7b92d"
server_key_version: "encryptionkey@0"
I20260808 02:45:14.708024 9277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "3e694296935a4daba8cf9be03f5034f1"
format_stamp: "Formatted at 2026-08-08 02:45:14 on dist-test-slave-k5rr"
server_key: "a96b5259aa7b7c9a8e8bc9a81ad295fd"
server_key_iv: "cb6ec8aad0b89158b9165618ccb7b92d"
server_key_version: "encryptionkey@0"
I20260808 02:45:14.713658 9277 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:45:14.717469 9299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:14.718883 9277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:45:14.719043 9277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "3e694296935a4daba8cf9be03f5034f1"
format_stamp: "Formatted at 2026-08-08 02:45:14 on dist-test-slave-k5rr"
server_key: "a96b5259aa7b7c9a8e8bc9a81ad295fd"
server_key_iv: "cb6ec8aad0b89158b9165618ccb7b92d"
server_key_version: "encryptionkey@0"
I20260808 02:45:14.719249 9277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:14.724903 9277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:14.725694 9277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:14.725945 9277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:14.726871 9277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:45:14.728298 9277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:45:14.728401 9277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:14.728509 9277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:45:14.728578 9277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:14.760854 9277 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:45801
I20260808 02:45:14.760908 9474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:45801 every 8 connection(s)
I20260808 02:45:14.762151 9277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:45:14.769001 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 9277
I20260808 02:45:14.769199 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:45:14.769778 26151 external_mini_cluster.cc:1428] Setting key 83417873805156b0a4a1e38230f8bfd7
I20260808 02:45:14.770928 9275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:14.764685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:36369 (local address 127.25.137.254:41781)
0808 02:45:14.764858 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:14.764862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:14.766454 (+ 1592us) server_negotiation.cc:407] Connection header received
0808 02:45:14.768232 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:14.768238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:14.768299 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:14.768398 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:14.769984 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:14.769989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:14.769992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:14.770013 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:14.770050 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:14.770064 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:14.770182 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:14.770482 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 02:45:14.770559 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:45:14.771945 9483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:14.765314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41781 (local address 127.25.137.194:36369)
0808 02:45:14.766289 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:14.766335 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:14.767946 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:14.768701 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:14.768712 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:14.768846 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:14.769611 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:45:14.769615 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:14.769664 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:45:14.769778 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:45:14.770184 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:45:14.770194 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:45:14.770202 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:45:14.770480 (+ 278us) client_negotiation.cc:240] Negotiation successful
0808 02:45:14.770707 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":285,"threads_started":1}
I20260808 02:45:14.773298 9480 heartbeater.cc:347] Connected to a master server at 127.25.137.254:41781
I20260808 02:45:14.773638 9480 heartbeater.cc:464] Registering TS with master...
I20260808 02:45:14.774269 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41781
--builtin_ntp_servers=127.25.137.212:45813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 02:45:14.774403 9480 heartbeater.cc:511] Master 127.25.137.254:41781 requested a full tablet report, sending...
I20260808 02:45:14.775931 8979 ts_manager.cc:194] Registered new tserver with Master: 3e694296935a4daba8cf9be03f5034f1 (127.25.137.194:45801)
W20260808 02:45:15.108767 9484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:15.109084 9484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:15.109229 9484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:15.116806 9484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:15.116909 9484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:15.117089 9484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:45:15.126643 9484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:15.127945 9484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:15.129740 9484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:15.142469 9497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:15.142403 9493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:15.142699 9492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:15.143702 9484 server_base.cc:1034] running on GCE node
I20260808 02:45:15.144346 9484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:15.145363 9484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:15.146591 9484 hybrid_clock.cc:648] HybridClock initialized: now 1786157115146544 us; error 38 us; skew 500 ppm
I20260808 02:45:15.146951 9484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:45:15.149449 9484 webserver.cc:460] Webserver started at http://127.25.137.195:40437/ using document root <none> and password file <none>
I20260808 02:45:15.150153 9484 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:15.150274 9484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:15.150638 9484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:15.153874 9484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "1a8292ce5f554368b85deb4a840393fa"
format_stamp: "Formatted at 2026-08-08 02:45:15 on dist-test-slave-k5rr"
server_key: "53f30a996ed5dda9ab92c28a4addac13"
server_key_iv: "4feae13faf90091d13abdd20632f36c3"
server_key_version: "encryptionkey@0"
I20260808 02:45:15.154726 9484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "1a8292ce5f554368b85deb4a840393fa"
format_stamp: "Formatted at 2026-08-08 02:45:15 on dist-test-slave-k5rr"
server_key: "53f30a996ed5dda9ab92c28a4addac13"
server_key_iv: "4feae13faf90091d13abdd20632f36c3"
server_key_version: "encryptionkey@0"
I20260808 02:45:15.160094 9484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 02:45:15.163705 9506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:15.164991 9484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:45:15.165225 9484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "1a8292ce5f554368b85deb4a840393fa"
format_stamp: "Formatted at 2026-08-08 02:45:15 on dist-test-slave-k5rr"
server_key: "53f30a996ed5dda9ab92c28a4addac13"
server_key_iv: "4feae13faf90091d13abdd20632f36c3"
server_key_version: "encryptionkey@0"
I20260808 02:45:15.165457 9484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:15.171032 9484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:15.171880 9484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:15.172137 9484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:15.173086 9484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:45:15.174503 9484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:45:15.174599 9484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:15.174708 9484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:45:15.174780 9484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:15.205202 9484 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:40183
I20260808 02:45:15.205284 9674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:40183 every 8 connection(s)
I20260808 02:45:15.206497 9484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:45:15.213052 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 9484
I20260808 02:45:15.213256 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:45:15.213745 26151 external_mini_cluster.cc:1428] Setting key 79d920b344fff78381b8e8a060f78639
I20260808 02:45:15.215206 9275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:15.209207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:59879 (local address 127.25.137.254:41781)
0808 02:45:15.209409 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:15.209415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:15.210855 (+ 1440us) server_negotiation.cc:407] Connection header received
0808 02:45:15.212672 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:15.212679 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:15.212836 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:15.212967 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:15.214457 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:15.214462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:15.214465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:15.214484 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:15.214520 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:15.214534 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:15.214676 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:15.214888 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 02:45:15.214940 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:45:15.216367 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:15.209704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41781 (local address 127.25.137.195:59879)
0808 02:45:15.210699 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:15.210743 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:15.212372 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:15.213191 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:15.213203 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:15.213350 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:15.214054 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:45:15.214057 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:15.214110 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:45:15.214229 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:45:15.214639 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:45:15.214652 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:45:15.214660 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:45:15.214957 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 02:45:15.215220 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":297,"threads_started":1}
I20260808 02:45:15.217797 9675 heartbeater.cc:347] Connected to a master server at 127.25.137.254:41781
I20260808 02:45:15.218079 9675 heartbeater.cc:464] Registering TS with master...
I20260808 02:45:15.218648 9675 heartbeater.cc:511] Master 127.25.137.254:41781 requested a full tablet report, sending...
I20260808 02:45:15.220027 8979 ts_manager.cc:194] Registered new tserver with Master: 1a8292ce5f554368b85deb4a840393fa (127.25.137.195:40183)
I20260808 02:45:15.229470 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:45:15.235936 9275 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:45:15.234282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54890 (local address 127.25.137.254:41781)
0808 02:45:15.234496 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:15.234503 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:15.235058 (+ 555us) server_negotiation.cc:407] Connection header received
0808 02:45:15.235213 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:15.235217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:15.235286 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:15.235358 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:15.235649 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54890: BlockingRecv error: recv got EOF from 127.0.0.1:54890 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 02:45:15.236413 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 9075
I20260808 02:45:15.240141 9690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:15.237618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48992 (local address 127.25.137.193:41811)
0808 02:45:15.238162 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:15.238168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:15.238193 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:45:15.238249 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:15.238255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:15.238452 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:15.238570 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:15.239481 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:15.239488 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:15.239499 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:15.239511 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:15.239559 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:15.239573 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:15.239674 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:15.239720 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 02:45:15.239843 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":290,"threads_started":1}
I20260808 02:45:15.241351 9261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:15.321687 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 9075
I20260808 02:45:15.344695 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 9277
I20260808 02:45:15.348237 9692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:15.345737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36564 (local address 127.25.137.194:45801)
0808 02:45:15.346378 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:15.346385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:15.346413 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:45:15.346478 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:15.346485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:15.346696 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:15.346831 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:15.347649 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:15.347655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:15.347666 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:15.347685 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:15.347726 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:15.347739 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:15.347847 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:15.347898 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 02:45:15.347976 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":297,"threads_started":1}
I20260808 02:45:15.349682 9469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:15.429934 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 9277
I20260808 02:45:15.452579 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 9484
I20260808 02:45:15.456435 9694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:15.453641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49336 (local address 127.25.137.195:40183)
0808 02:45:15.454348 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:15.454357 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:15.454389 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:45:15.454455 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:15.454461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:15.454676 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:15.454805 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:15.455705 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:15.455710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:15.455721 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:45:15.455738 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:15.455776 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:45:15.455789 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:15.455961 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:15.456016 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 02:45:15.456104 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":385,"threads_started":1}
I20260808 02:45:15.457871 9667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:15.544463 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 9484
I20260808 02:45:15.566067 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 8929
I20260808 02:45:15.567381 9022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:15.670023 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 8929
2026-08-08T02:45:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2406 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:45:15 dist-test-slave-k5rr krb5kdc[9705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:45:15 dist-test-slave-k5rr krb5kdc[9705](info): set up 2 sockets
Aug 08 02:45:15 dist-test-slave-k5rr krb5kdc[9705](info): commencing operation
krb5kdc: starting...
W20260808 02:45:16.895751 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:45:17 dist-test-slave-k5rr krb5kdc[9705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:45:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:45:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:45:17.184286 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:36031
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:36031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:45:17.525278 9723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:17.525569 9723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:17.525707 9723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:17.532706 9723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:45:17.532823 9723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:17.532914 9723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:17.532996 9723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:45:17.533072 9723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:45:17.542598 9723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:36031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:36031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:17.543874 9723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:17.545161 9723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:17.556584 9729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:17.556640 9734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:17.556687 9731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:17.557407 9723 server_base.cc:1034] running on GCE node
I20260808 02:45:17.558149 9723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:17.559511 9723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:17.560721 9723 hybrid_clock.cc:648] HybridClock initialized: now 1786157117560663 us; error 50 us; skew 500 ppm
I20260808 02:45:17.561175 9723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:45:17 dist-test-slave-k5rr krb5kdc[9705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157117, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:45:17.567221 9723 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:45:17.568645 9723 webserver.cc:460] Webserver started at http://127.25.137.254:43749/ using document root <none> and password file <none>
I20260808 02:45:17.569332 9723 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:17.569437 9723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:17.569679 9723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:17.572611 9723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "5579bd139948445b83a9c052d38173bd"
format_stamp: "Formatted at 2026-08-08 02:45:17 on dist-test-slave-k5rr"
server_key: "b9e6d967472636ba38afb2fdd3c69c22"
server_key_iv: "e5b0477289c9d53c18f56bfa7f41c3e9"
server_key_version: "encryptionkey@0"
I20260808 02:45:17.573375 9723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "5579bd139948445b83a9c052d38173bd"
format_stamp: "Formatted at 2026-08-08 02:45:17 on dist-test-slave-k5rr"
server_key: "b9e6d967472636ba38afb2fdd3c69c22"
server_key_iv: "e5b0477289c9d53c18f56bfa7f41c3e9"
server_key_version: "encryptionkey@0"
I20260808 02:45:17.578346 9723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 02:45:17.581689 9743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:17.582952 9723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:45:17.583106 9723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "5579bd139948445b83a9c052d38173bd"
format_stamp: "Formatted at 2026-08-08 02:45:17 on dist-test-slave-k5rr"
server_key: "b9e6d967472636ba38afb2fdd3c69c22"
server_key_iv: "e5b0477289c9d53c18f56bfa7f41c3e9"
server_key_version: "encryptionkey@0"
I20260808 02:45:17.583271 9723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:17.600346 9723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:17.604225 9723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:17.604487 9723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:17.621970 9723 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:36031
I20260808 02:45:17.622020 9815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:36031 every 8 connection(s)
I20260808 02:45:17.623320 9723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:45:17.627199 9818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:45:17.632329 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 9723
I20260808 02:45:17.632519 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:45:17.633210 26151 external_mini_cluster.cc:1428] Setting key 93ccf34d6d0c1c90128598d7f9ecb608
I20260808 02:45:17.637807 9818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd: Bootstrap starting.
I20260808 02:45:17.641471 9818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd: Neither blocks nor log segments found. Creating new log.
I20260808 02:45:17.642663 9818 log.cc:824] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd: Log is configured to *not* fsync() on all Append() calls
I20260808 02:45:17.645655 9818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd: No bootstrap required, opened a new log
Aug 08 02:45:17 dist-test-slave-k5rr krb5kdc[9705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:45:17.653661 9818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5579bd139948445b83a9c052d38173bd" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36031 } }
I20260808 02:45:17.654014 9818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:45:17.654182 9818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5579bd139948445b83a9c052d38173bd, State: Initialized, Role: FOLLOWER
I20260808 02:45:17.654868 9818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5579bd139948445b83a9c052d38173bd" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36031 } }
I20260808 02:45:17.655153 9818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:45:17.655316 9818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:45:17.655499 9818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:45:17.655987 9824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:17.635110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.25.137.254:36031)
0808 02:45:17.635913 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:17.635928 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:17.635968 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:45:17.637802 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:17.637826 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:17.638251 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:17.638585 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:17.640459 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:17.641850 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:17.643610 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:17.644300 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:17.644468 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:17.648594 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:17.648605 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:17.648620 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:45:17.648664 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:17.651708 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:17.652377 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:17.652384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:17.652391 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:17.652562 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:17.652933 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:17.652937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:17.652939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:17.653529 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:17.653836 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:17.654378 (+ 542us) server_negotiation.cc:299] Negotiation successful
0808 02:45:17.654775 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":372,"threads_started":1}
I20260808 02:45:17.658699 9818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5579bd139948445b83a9c052d38173bd" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36031 } }
I20260808 02:45:17.659268 9818 leader_election.cc:302] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5579bd139948445b83a9c052d38173bd; no voters:
I20260808 02:45:17.659924 9818 leader_election.cc:288] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:45:17.660039 9826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:45:17.660662 9826 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [term 1 LEADER]: Becoming Leader. State: Replica: 5579bd139948445b83a9c052d38173bd, State: Running, Role: LEADER
I20260808 02:45:17.661201 9826 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5579bd139948445b83a9c052d38173bd" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36031 } }
I20260808 02:45:17.661942 9818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:45:17.663892 9827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5579bd139948445b83a9c052d38173bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5579bd139948445b83a9c052d38173bd" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36031 } } }
I20260808 02:45:17.664245 9827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [sys.catalog]: This master's current role is: LEADER
I20260808 02:45:17.665278 9828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5579bd139948445b83a9c052d38173bd. Latest consensus state: current_term: 1 leader_uuid: "5579bd139948445b83a9c052d38173bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5579bd139948445b83a9c052d38173bd" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36031 } } }
I20260808 02:45:17.665971 9828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd [sys.catalog]: This master's current role is: LEADER
I20260808 02:45:17.670905 9839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:45:17.675302 9839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:45:17.683139 9839 catalog_manager.cc:1347] Generated new cluster ID: 98939fbb408743db8120850c76a9bb15
I20260808 02:45:17.683256 9839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:45:17.706498 9839 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:45:17.707792 9839 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:45:17.718190 9839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd: Generated new TSK 0
I20260808 02:45:17.719223 9839 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:45:17.789764 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:36031
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:45:18.122102 9857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:18.122376 9857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:18.122498 9857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:18.129760 9857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:18.129864 9857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:18.130019 9857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:45:18.139707 9857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:18.140952 9857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:18.142292 9857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:18.154721 9869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:18.154784 9866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:18.154999 9863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:18.155807 9857 server_base.cc:1034] running on GCE node
I20260808 02:45:18.156430 9857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:18.157378 9857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:18.158584 9857 hybrid_clock.cc:648] HybridClock initialized: now 1786157118158540 us; error 41 us; skew 500 ppm
I20260808 02:45:18.158874 9857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:45:18 dist-test-slave-k5rr krb5kdc[9705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157118, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:45:18.161924 9857 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:45:18.163357 9857 webserver.cc:460] Webserver started at http://127.25.137.193:44425/ using document root <none> and password file <none>
I20260808 02:45:18.163986 9857 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:18.164088 9857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:18.164353 9857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:18.167418 9857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "193413835bd042d0b8e50145424aeb67"
format_stamp: "Formatted at 2026-08-08 02:45:18 on dist-test-slave-k5rr"
server_key: "0cac3679494015b158ffe0224fc8b746"
server_key_iv: "8fb0d9354e2c2bfa15fa72857a47b784"
server_key_version: "encryptionkey@0"
I20260808 02:45:18.168207 9857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "193413835bd042d0b8e50145424aeb67"
format_stamp: "Formatted at 2026-08-08 02:45:18 on dist-test-slave-k5rr"
server_key: "0cac3679494015b158ffe0224fc8b746"
server_key_iv: "8fb0d9354e2c2bfa15fa72857a47b784"
server_key_version: "encryptionkey@0"
I20260808 02:45:18.173524 9857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 02:45:18.176987 9878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:18.178324 9857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:45:18.178484 9857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "193413835bd042d0b8e50145424aeb67"
format_stamp: "Formatted at 2026-08-08 02:45:18 on dist-test-slave-k5rr"
server_key: "0cac3679494015b158ffe0224fc8b746"
server_key_iv: "8fb0d9354e2c2bfa15fa72857a47b784"
server_key_version: "encryptionkey@0"
I20260808 02:45:18.178658 9857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:18.215716 9857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:18.219992 9857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:18.220273 9857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:18.221244 9857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:45:18.222569 9857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:45:18.222648 9857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:18.222774 9857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:45:18.222844 9857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:18.253482 9857 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:34809
I20260808 02:45:18.253549 10052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:34809 every 8 connection(s)
I20260808 02:45:18.254743 9857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:45:18.258256 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 9857
I20260808 02:45:18.258508 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:45:18.259335 26151 external_mini_cluster.cc:1428] Setting key 26861c53636a3f9b72d5ca0865e29d6c
Aug 08 02:45:18 dist-test-slave-k5rr krb5kdc[9705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157118, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:45:18.274286 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:18.257365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:35009 (local address 127.25.137.254:36031)
0808 02:45:18.257822 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:18.257826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:18.258968 (+ 1142us) server_negotiation.cc:407] Connection header received
0808 02:45:18.260883 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:18.260888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:18.260967 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:18.261088 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:18.262638 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:18.263400 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:18.264926 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:18.265362 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:18.265477 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:18.268850 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:18.268855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:18.268858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:45:18.268890 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:18.271307 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:18.272011 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:18.272016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:18.272019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:18.272135 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:18.272550 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:18.272555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:18.272557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:18.272816 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:18.272956 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:18.273830 (+ 874us) server_negotiation.cc:299] Negotiation successful
0808 02:45:18.274019 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":214,"threads_started":1}
I20260808 02:45:18.275373 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:18.257979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:36031 (local address 127.25.137.193:35009)
0808 02:45:18.258782 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:18.258858 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:18.260633 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:18.261325 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:18.261333 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:18.261886 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:18.262470 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:18.262487 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:18.263549 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:18.263553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:18.264768 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:18.264781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:18.265594 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:18.265601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:18.265849 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:18.266454 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:18.266482 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:18.268637 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:45:18.271450 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:45:18.271459 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:45:18.271474 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:45:18.271894 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:45:18.272249 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:45:18.272252 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:45:18.272254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:45:18.272442 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:45:18.272954 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:45:18.272962 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:45:18.273367 (+ 405us) client_negotiation.cc:769] Sending connection context
0808 02:45:18.273679 (+ 312us) client_negotiation.cc:240] Negotiation successful
0808 02:45:18.274210 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":342,"threads_started":1}
I20260808 02:45:18.276862 10053 heartbeater.cc:347] Connected to a master server at 127.25.137.254:36031
I20260808 02:45:18.277207 10053 heartbeater.cc:464] Registering TS with master...
I20260808 02:45:18.278072 10053 heartbeater.cc:511] Master 127.25.137.254:36031 requested a full tablet report, sending...
I20260808 02:45:18.280380 9764 ts_manager.cc:194] Registered new tserver with Master: 193413835bd042d0b8e50145424aeb67 (127.25.137.193:34809)
I20260808 02:45:18.281894 9764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:35009
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:45:18.323364 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:36031
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:45:18.654249 10060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:18.654552 10060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:18.654685 10060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:18.662848 10060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:18.662967 10060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:18.663128 10060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:45:18.672955 10060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:18.674284 10060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:18.675555 10060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:18.688340 10066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:18.688410 10067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:18.688545 10069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:18.689442 10060 server_base.cc:1034] running on GCE node
I20260808 02:45:18.690089 10060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:18.691071 10060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:18.692317 10060 hybrid_clock.cc:648] HybridClock initialized: now 1786157118692244 us; error 52 us; skew 500 ppm
I20260808 02:45:18.692648 10060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:45:18 dist-test-slave-k5rr krb5kdc[9705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157118, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:45:18.695844 10060 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:45:18.697484 10060 webserver.cc:460] Webserver started at http://127.25.137.194:34559/ using document root <none> and password file <none>
I20260808 02:45:18.698201 10060 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:18.698345 10060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:18.698794 10060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:18.702019 10060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "a3ac2c59fea84533939ed58dbba629f9"
format_stamp: "Formatted at 2026-08-08 02:45:18 on dist-test-slave-k5rr"
server_key: "70d8f6cfd89482848c6d72a1f1b33e08"
server_key_iv: "ff8b9d8543b766f66135708c8ce8722a"
server_key_version: "encryptionkey@0"
I20260808 02:45:18.702890 10060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "a3ac2c59fea84533939ed58dbba629f9"
format_stamp: "Formatted at 2026-08-08 02:45:18 on dist-test-slave-k5rr"
server_key: "70d8f6cfd89482848c6d72a1f1b33e08"
server_key_iv: "ff8b9d8543b766f66135708c8ce8722a"
server_key_version: "encryptionkey@0"
I20260808 02:45:18.708599 10060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:45:18.712386 10078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:18.713889 10060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:45:18.714071 10060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "a3ac2c59fea84533939ed58dbba629f9"
format_stamp: "Formatted at 2026-08-08 02:45:18 on dist-test-slave-k5rr"
server_key: "70d8f6cfd89482848c6d72a1f1b33e08"
server_key_iv: "ff8b9d8543b766f66135708c8ce8722a"
server_key_version: "encryptionkey@0"
I20260808 02:45:18.714283 10060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:18.729454 10060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:18.733569 10060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:18.733884 10060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:18.734900 10060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:45:18.736368 10060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:45:18.736457 10060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:18.736590 10060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:45:18.736658 10060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:18.769306 10060 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:42783
I20260808 02:45:18.769327 10250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:42783 every 8 connection(s)
I20260808 02:45:18.770656 10060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:45:18.780166 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 10060
I20260808 02:45:18.780328 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:45:18.780769 26151 external_mini_cluster.cc:1428] Setting key 5af2dce5f2bea8aea647588bdb991422
Aug 08 02:45:18 dist-test-slave-k5rr krb5kdc[9705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157118, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:45:18.790591 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:18.773124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:37215 (local address 127.25.137.254:36031)
0808 02:45:18.773317 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:18.773321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:18.774486 (+ 1165us) server_negotiation.cc:407] Connection header received
0808 02:45:18.776415 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:18.776422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:18.776508 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:18.776607 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:18.778195 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:18.778940 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:18.780554 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:18.781075 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:18.781205 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:18.784925 (+ 3720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:18.784931 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:18.784934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:45:18.784967 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:18.787706 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:18.788360 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:18.788363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:18.788366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:18.788490 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:18.788912 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:18.788918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:18.788920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:18.789223 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:18.789360 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:18.790019 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 02:45:18.790241 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 02:45:18.791465 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:18.773570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:36031 (local address 127.25.137.194:37215)
0808 02:45:18.774313 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:18.774352 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:18.776133 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:18.776867 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:18.776875 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:18.777434 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:18.778012 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:18.778030 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:18.779103 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:18.779106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:18.780406 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:18.780424 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:18.781300 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:18.781309 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:18.781584 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:18.782349 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:18.782373 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:18.784685 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:45:18.787842 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:45:18.787851 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:45:18.787866 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:45:18.788247 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:45:18.788592 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:45:18.788597 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:45:18.788602 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:45:18.788807 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:45:18.789349 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:45:18.789356 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:45:18.789669 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 02:45:18.789902 (+ 233us) client_negotiation.cc:240] Negotiation successful
0808 02:45:18.790382 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":275,"threads_started":1}
I20260808 02:45:18.792709 10251 heartbeater.cc:347] Connected to a master server at 127.25.137.254:36031
I20260808 02:45:18.793047 10251 heartbeater.cc:464] Registering TS with master...
I20260808 02:45:18.793876 10251 heartbeater.cc:511] Master 127.25.137.254:36031 requested a full tablet report, sending...
I20260808 02:45:18.795380 9764 ts_manager.cc:194] Registered new tserver with Master: a3ac2c59fea84533939ed58dbba629f9 (127.25.137.194:42783)
I20260808 02:45:18.796247 9764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:37215
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:45:18.839982 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:36031
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:45:19.169564 10258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:45:19.169855 10258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:45:19.169975 10258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:45:19.177199 10258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:45:19.177304 10258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:45:19.177454 10258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:45:19.187225 10258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:36031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:45:19.188542 10258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:45:19.189826 10258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:45:19.202415 10265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:19.202454 10268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:45:19.202502 10266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:45:19.203667 10258 server_base.cc:1034] running on GCE node
I20260808 02:45:19.204257 10258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:45:19.205193 10258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:45:19.206404 10258 hybrid_clock.cc:648] HybridClock initialized: now 1786157119206356 us; error 34 us; skew 500 ppm
I20260808 02:45:19.206702 10258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:45:19 dist-test-slave-k5rr krb5kdc[9705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157119, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:45:19.209854 10258 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:45:19.211302 10258 webserver.cc:460] Webserver started at http://127.25.137.195:37477/ using document root <none> and password file <none>
I20260808 02:45:19.211990 10258 fs_manager.cc:359] Metadata directory not provided
I20260808 02:45:19.212096 10258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:45:19.212355 10258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:45:19.215343 10258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "510046c57366489a99a8b300c0b567ff"
format_stamp: "Formatted at 2026-08-08 02:45:19 on dist-test-slave-k5rr"
server_key: "ea4fc568a4569b30a8c9e6f0d9301ab6"
server_key_iv: "d6ecf2a75d9ef7475d35652adfcd7e37"
server_key_version: "encryptionkey@0"
I20260808 02:45:19.216140 10258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "510046c57366489a99a8b300c0b567ff"
format_stamp: "Formatted at 2026-08-08 02:45:19 on dist-test-slave-k5rr"
server_key: "ea4fc568a4569b30a8c9e6f0d9301ab6"
server_key_iv: "d6ecf2a75d9ef7475d35652adfcd7e37"
server_key_version: "encryptionkey@0"
I20260808 02:45:19.221431 10258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 02:45:19.224646 10280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:19.226025 10258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:45:19.226184 10258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "510046c57366489a99a8b300c0b567ff"
format_stamp: "Formatted at 2026-08-08 02:45:19 on dist-test-slave-k5rr"
server_key: "ea4fc568a4569b30a8c9e6f0d9301ab6"
server_key_iv: "d6ecf2a75d9ef7475d35652adfcd7e37"
server_key_version: "encryptionkey@0"
I20260808 02:45:19.226356 10258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:45:19.241503 10258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:45:19.245573 10258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:45:19.245851 10258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:45:19.246834 10258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:45:19.248162 10258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:45:19.248238 10258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:19.248413 10258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:45:19.248507 10258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:45:19.279381 10258 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:35309
I20260808 02:45:19.279417 10462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:35309 every 8 connection(s)
I20260808 02:45:19.280573 10258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:45:19.285279 10053 heartbeater.cc:503] Master 127.25.137.254:36031 was elected leader, sending a full tablet report...
I20260808 02:45:19.287528 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 10258
I20260808 02:45:19.287680 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:45:19.288218 26151 external_mini_cluster.cc:1428] Setting key c065ef428e7cb11a82e3ccdaf31a309c
Aug 08 02:45:19 dist-test-slave-k5rr krb5kdc[9705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157119, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:45:19.300010 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.283153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:38263 (local address 127.25.137.254:36031)
0808 02:45:19.283326 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:19.283330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:19.284439 (+ 1109us) server_negotiation.cc:407] Connection header received
0808 02:45:19.286520 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:19.286525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:19.286623 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:19.286726 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:19.288289 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.289201 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.291262 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.291687 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.291792 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.295051 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:19.295056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:19.295059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:45:19.295089 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:19.297513 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:19.298159 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:19.298163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:19.298166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:19.298317 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:19.298644 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:19.298647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:19.298649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:19.298898 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:19.299007 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:19.299589 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 02:45:19.299760 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:45:19.300920 10470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.283569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:36031 (local address 127.25.137.195:38263)
0808 02:45:19.284286 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:19.284325 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:19.286230 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:19.286959 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:19.286968 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:19.287545 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:45:19.288129 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.288147 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.289361 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.289366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.291097 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.291112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.291894 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.291902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.292175 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.292761 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:45:19.292792 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:45:19.294870 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:45:19.297632 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:45:19.297638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:45:19.297649 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:45:19.298053 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:45:19.298398 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:45:19.298401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:45:19.298403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:45:19.298568 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:45:19.298991 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:45:19.298996 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:45:19.299291 (+ 295us) client_negotiation.cc:769] Sending connection context
0808 02:45:19.299504 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 02:45:19.299856 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":267,"threads_started":1}
I20260808 02:45:19.302367 10467 heartbeater.cc:347] Connected to a master server at 127.25.137.254:36031
I20260808 02:45:19.302663 10467 heartbeater.cc:464] Registering TS with master...
I20260808 02:45:19.303483 10467 heartbeater.cc:511] Master 127.25.137.254:36031 requested a full tablet report, sending...
I20260808 02:45:19.304713 9764 ts_manager.cc:194] Registered new tserver with Master: 510046c57366489a99a8b300c0b567ff (127.25.137.195:35309)
I20260808 02:45:19.305506 9764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:38263
I20260808 02:45:19.315420 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:45:19.332228 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.319861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.25.137.254:36031)
0808 02:45:19.320050 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:19.320057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:19.320510 (+ 453us) server_negotiation.cc:407] Connection header received
0808 02:45:19.320683 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:19.320687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:19.320749 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:19.320873 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:19.322524 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.323271 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.324474 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.324897 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.325041 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.327262 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:19.327267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:19.327271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:45:19.327303 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:19.329627 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:19.330236 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:19.330242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:19.330245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:19.330396 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:19.330747 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:19.330760 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:19.330764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:19.331102 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:19.331358 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:19.331698 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 02:45:19.331919 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:45:19.336236 9764 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:45:19.343148 9764 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:45:19.374014 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.355410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57092 (local address 127.25.137.194:42783)
0808 02:45:19.365165 (+ 9755us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:19.365176 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:19.365211 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:45:19.365333 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:19.365342 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:19.366803 (+ 1461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:19.367029 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:45:19.368005 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.369469 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.371948 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.372875 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.373052 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.373238 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:19.373546 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 02:45:19.373715 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9601,"thread_start_us":9490,"threads_started":1}
I20260808 02:45:19.374123 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.355155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:42783 (local address 127.0.0.1:57092)
0808 02:45:19.355829 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:19.355867 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:19.356058 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:19.367346 (+ 11288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:19.367357 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:19.367392 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:45:19.367810 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.367822 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.369629 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.369633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.371783 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.371796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.373119 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.373125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.373331 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.373369 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 02:45:19.373485 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 02:45:19.373678 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":3,"thread_start_us":298,"threads_started":1}
I20260808 02:45:19.375869 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.356709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45402 (local address 127.25.137.195:35309)
0808 02:45:19.368477 (+ 11768us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:19.368485 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:19.368518 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:45:19.368606 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:19.368613 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:19.368838 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:19.369054 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:45:19.369681 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.371053 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.373638 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.374810 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.374986 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.375151 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:19.375379 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 02:45:19.375535 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11594,"thread_start_us":11439,"threads_started":1}
I20260808 02:45:19.376684 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.356449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34809 (local address 127.0.0.1:37446)
0808 02:45:19.368773 (+ 12324us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:19.368789 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:19.368905 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:19.369529 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:19.369533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:19.369559 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:45:19.370074 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.370088 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.372004 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.372009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.374332 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.374348 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.375883 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.375890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.376045 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.376077 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:45:19.376159 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 02:45:19.376280 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12214,"mutex_wait_us":22,"thread_start_us":8123,"threads_started":1}
I20260808 02:45:19.376801 10501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.356310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:35309 (local address 127.0.0.1:45402)
0808 02:45:19.364870 (+ 8560us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:19.364892 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:19.365040 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:19.369054 (+ 4014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:19.369058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:19.369080 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:45:19.369467 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.369481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.371206 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.371211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.373455 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.373471 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.374991 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.374997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.375144 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.375187 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:45:19.375263 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:45:19.376453 (+ 1190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8414,"thread_start_us":12226,"threads_started":1}
I20260808 02:45:19.377483 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.357144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37446 (local address 127.25.137.193:34809)
0808 02:45:19.368113 (+ 10969us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:19.368122 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:19.368893 (+ 771us) server_negotiation.cc:407] Connection header received
0808 02:45:19.369059 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:19.369066 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:19.369306 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:19.369515 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:45:19.370280 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.371823 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.374489 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.375686 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.375884 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.376040 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:19.376217 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 02:45:19.376971 (+ 754us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10788,"thread_start_us":10581,"threads_started":1}
I20260808 02:45:19.378564 10155 tablet_service.cc:1467] Processing CreateTablet for tablet 91758e7c78aa4f16a81ad8b8b9549c61 (DEFAULT_TABLE table=test-table [id=379938c22dc541818a21b1943d03ae37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:45:19.379868 10155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91758e7c78aa4f16a81ad8b8b9549c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:45:19.380784 10362 tablet_service.cc:1467] Processing CreateTablet for tablet 91758e7c78aa4f16a81ad8b8b9549c61 (DEFAULT_TABLE table=test-table [id=379938c22dc541818a21b1943d03ae37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:45:19.380869 9955 tablet_service.cc:1467] Processing CreateTablet for tablet 91758e7c78aa4f16a81ad8b8b9549c61 (DEFAULT_TABLE table=test-table [id=379938c22dc541818a21b1943d03ae37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:45:19.382266 9955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91758e7c78aa4f16a81ad8b8b9549c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:45:19.382256 10362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91758e7c78aa4f16a81ad8b8b9549c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:45:19.390712 10506 tablet_bootstrap.cc:492] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: Bootstrap starting.
I20260808 02:45:19.394315 10506 tablet_bootstrap.cc:654] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: Neither blocks nor log segments found. Creating new log.
I20260808 02:45:19.395344 10506 log.cc:824] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:45:19.396780 10508 tablet_bootstrap.cc:492] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: Bootstrap starting.
I20260808 02:45:19.398193 10510 tablet_bootstrap.cc:492] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: Bootstrap starting.
I20260808 02:45:19.399119 10506 tablet_bootstrap.cc:492] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: No bootstrap required, opened a new log
I20260808 02:45:19.399412 10506 ts_tablet_manager.cc:1397] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 02:45:19.400996 10510 tablet_bootstrap.cc:654] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: Neither blocks nor log segments found. Creating new log.
I20260808 02:45:19.401316 10508 tablet_bootstrap.cc:654] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: Neither blocks nor log segments found. Creating new log.
I20260808 02:45:19.402052 10510 log.cc:824] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: Log is configured to *not* fsync() on all Append() calls
I20260808 02:45:19.402454 10508 log.cc:824] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: Log is configured to *not* fsync() on all Append() calls
I20260808 02:45:19.405133 10510 tablet_bootstrap.cc:492] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: No bootstrap required, opened a new log
I20260808 02:45:19.405411 10510 ts_tablet_manager.cc:1397] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260808 02:45:19.406571 10508 tablet_bootstrap.cc:492] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: No bootstrap required, opened a new log
I20260808 02:45:19.406829 10508 ts_tablet_manager.cc:1397] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260808 02:45:19.409309 10506 raft_consensus.cc:348] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3ac2c59fea84533939ed58dbba629f9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42783 } } peers { permanent_uuid: "510046c57366489a99a8b300c0b567ff" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 35309 } } peers { permanent_uuid: "193413835bd042d0b8e50145424aeb67" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34809 } }
I20260808 02:45:19.409749 10506 raft_consensus.cc:374] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:45:19.409890 10506 raft_consensus.cc:729] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ac2c59fea84533939ed58dbba629f9, State: Initialized, Role: FOLLOWER
I20260808 02:45:19.410508 10506 consensus_queue.cc:260] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3ac2c59fea84533939ed58dbba629f9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42783 } } peers { permanent_uuid: "510046c57366489a99a8b300c0b567ff" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 35309 } } peers { permanent_uuid: "193413835bd042d0b8e50145424aeb67" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34809 } }
I20260808 02:45:19.411702 10251 heartbeater.cc:503] Master 127.25.137.254:36031 was elected leader, sending a full tablet report...
I20260808 02:45:19.412770 10510 raft_consensus.cc:348] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3ac2c59fea84533939ed58dbba629f9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42783 } } peers { permanent_uuid: "510046c57366489a99a8b300c0b567ff" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 35309 } } peers { permanent_uuid: "193413835bd042d0b8e50145424aeb67" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34809 } }
I20260808 02:45:19.413198 10510 raft_consensus.cc:374] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:45:19.413317 10510 raft_consensus.cc:729] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193413835bd042d0b8e50145424aeb67, State: Initialized, Role: FOLLOWER
I20260808 02:45:19.413647 10506 ts_tablet_manager.cc:1428] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260808 02:45:19.414013 10510 consensus_queue.cc:260] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3ac2c59fea84533939ed58dbba629f9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42783 } } peers { permanent_uuid: "510046c57366489a99a8b300c0b567ff" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 35309 } } peers { permanent_uuid: "193413835bd042d0b8e50145424aeb67" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34809 } }
I20260808 02:45:19.415787 10510 ts_tablet_manager.cc:1428] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 02:45:19.416594 10508 raft_consensus.cc:348] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3ac2c59fea84533939ed58dbba629f9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42783 } } peers { permanent_uuid: "510046c57366489a99a8b300c0b567ff" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 35309 } } peers { permanent_uuid: "193413835bd042d0b8e50145424aeb67" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34809 } }
I20260808 02:45:19.417043 10508 raft_consensus.cc:374] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:45:19.417155 10508 raft_consensus.cc:729] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 510046c57366489a99a8b300c0b567ff, State: Initialized, Role: FOLLOWER
I20260808 02:45:19.417709 10508 consensus_queue.cc:260] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3ac2c59fea84533939ed58dbba629f9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42783 } } peers { permanent_uuid: "510046c57366489a99a8b300c0b567ff" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 35309 } } peers { permanent_uuid: "193413835bd042d0b8e50145424aeb67" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34809 } }
I20260808 02:45:19.418991 10467 heartbeater.cc:503] Master 127.25.137.254:36031 was elected leader, sending a full tablet report...
I20260808 02:45:19.419250 10508 ts_tablet_manager.cc:1428] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: Time spent starting tablet: real 0.012s user 0.010s sys 0.003s
W20260808 02:45:19.508144 10054 tablet.cc:2369] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:45:19.522678 10252 tablet.cc:2369] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:45:19.532390 10468 tablet.cc:2369] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:45:19.544339 10514 raft_consensus.cc:482] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:45:19.544710 10514 raft_consensus.cc:504] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3ac2c59fea84533939ed58dbba629f9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42783 } } peers { permanent_uuid: "510046c57366489a99a8b300c0b567ff" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 35309 } } peers { permanent_uuid: "193413835bd042d0b8e50145424aeb67" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34809 } }
I20260808 02:45:19.546424 10514 leader_election.cc:288] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 510046c57366489a99a8b300c0b567ff (127.25.137.195:35309), 193413835bd042d0b8e50145424aeb67 (127.25.137.193:34809)
I20260808 02:45:19.553481 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.546929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:37643 (local address 127.25.137.193:34809)
0808 02:45:19.547148 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:19.547154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:19.547977 (+ 823us) server_negotiation.cc:407] Connection header received
0808 02:45:19.548147 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:19.548153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:19.548231 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:19.548310 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:45:19.549001 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.549767 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.551605 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.552439 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.552560 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.552611 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:19.553039 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 02:45:19.553174 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:45:19.553501 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.546841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:49109 (local address 127.25.137.195:35309)
0808 02:45:19.547065 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:19.547072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:19.548070 (+ 998us) server_negotiation.cc:407] Connection header received
0808 02:45:19.548226 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:19.548230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:19.548294 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:19.548391 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:45:19.549078 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.549800 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.551624 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.552594 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.552757 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.552828 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:19.553140 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 02:45:19.553275 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 02:45:19.553577 10518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.546915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34809 (local address 127.25.137.194:37643)
0808 02:45:19.547865 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:19.547881 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:19.548006 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:19.548342 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:19.548346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:19.548371 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:45:19.548845 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.548858 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.549896 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.549901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.551435 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.551447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.552654 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.552660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.552860 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.552891 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:45:19.552966 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:45:19.553102 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":6,"spinlock_wait_cycles":26368,"thread_start_us":615,"threads_started":1}
I20260808 02:45:19.553627 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.546972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:35309 (local address 127.25.137.194:49109)
0808 02:45:19.547948 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0808 02:45:19.547967 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:45:19.548105 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:45:19.548420 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:45:19.548424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:45:19.548453 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:45:19.548890 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.548899 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.549944 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.549951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.551458 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:45:19.551471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.552801 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.552806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:45:19.552965 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.552997 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:45:19.553108 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 02:45:19.553225 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":63,"spinlock_wait_cycles":25472,"thread_start_us":508,"threads_started":1}
I20260808 02:45:19.554702 10397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91758e7c78aa4f16a81ad8b8b9549c61" candidate_uuid: "a3ac2c59fea84533939ed58dbba629f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "510046c57366489a99a8b300c0b567ff" is_pre_election: true
I20260808 02:45:19.554702 9986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91758e7c78aa4f16a81ad8b8b9549c61" candidate_uuid: "a3ac2c59fea84533939ed58dbba629f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193413835bd042d0b8e50145424aeb67" is_pre_election: true
I20260808 02:45:19.555305 10397 raft_consensus.cc:2454] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3ac2c59fea84533939ed58dbba629f9 in term 0.
I20260808 02:45:19.555294 9986 raft_consensus.cc:2454] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3ac2c59fea84533939ed58dbba629f9 in term 0.
I20260808 02:45:19.556007 10081 leader_election.cc:302] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193413835bd042d0b8e50145424aeb67, a3ac2c59fea84533939ed58dbba629f9; no voters:
I20260808 02:45:19.556422 10514 raft_consensus.cc:2781] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:45:19.556553 10514 raft_consensus.cc:482] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:45:19.556648 10514 raft_consensus.cc:3037] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:45:19.558916 10514 raft_consensus.cc:504] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3ac2c59fea84533939ed58dbba629f9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42783 } } peers { permanent_uuid: "510046c57366489a99a8b300c0b567ff" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 35309 } } peers { permanent_uuid: "193413835bd042d0b8e50145424aeb67" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34809 } }
I20260808 02:45:19.559481 10514 leader_election.cc:288] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [CANDIDATE]: Term 1 election: Requested vote from peers 510046c57366489a99a8b300c0b567ff (127.25.137.195:35309), 193413835bd042d0b8e50145424aeb67 (127.25.137.193:34809)
I20260808 02:45:19.559921 10397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91758e7c78aa4f16a81ad8b8b9549c61" candidate_uuid: "a3ac2c59fea84533939ed58dbba629f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "510046c57366489a99a8b300c0b567ff"
I20260808 02:45:19.559957 9986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91758e7c78aa4f16a81ad8b8b9549c61" candidate_uuid: "a3ac2c59fea84533939ed58dbba629f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193413835bd042d0b8e50145424aeb67"
I20260808 02:45:19.560084 10397 raft_consensus.cc:3037] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:45:19.560130 9986 raft_consensus.cc:3037] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:45:19.562410 10397 raft_consensus.cc:2454] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3ac2c59fea84533939ed58dbba629f9 in term 1.
I20260808 02:45:19.562433 9986 raft_consensus.cc:2454] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3ac2c59fea84533939ed58dbba629f9 in term 1.
I20260808 02:45:19.562858 10080 leader_election.cc:302] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 510046c57366489a99a8b300c0b567ff, a3ac2c59fea84533939ed58dbba629f9; no voters:
I20260808 02:45:19.563175 10514 raft_consensus.cc:2781] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:45:19.563634 10514 raft_consensus.cc:686] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 1 LEADER]: Becoming Leader. State: Replica: a3ac2c59fea84533939ed58dbba629f9, State: Running, Role: LEADER
I20260808 02:45:19.564095 10514 consensus_queue.cc:237] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3ac2c59fea84533939ed58dbba629f9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42783 } } peers { permanent_uuid: "510046c57366489a99a8b300c0b567ff" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 35309 } } peers { permanent_uuid: "193413835bd042d0b8e50145424aeb67" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34809 } }
I20260808 02:45:19.568994 9764 catalog_manager.cc:5495] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3ac2c59fea84533939ed58dbba629f9 (127.25.137.194). New cstate: current_term: 1 leader_uuid: "a3ac2c59fea84533939ed58dbba629f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3ac2c59fea84533939ed58dbba629f9" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "510046c57366489a99a8b300c0b567ff" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 35309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193413835bd042d0b8e50145424aeb67" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34809 } health_report { overall_health: UNKNOWN } } }
I20260808 02:45:19.602190 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.596958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57094 (local address 127.25.137.194:42783)
0808 02:45:19.597203 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:19.597208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:19.597229 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:45:19.597363 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:19.597367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:19.597453 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:19.597579 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:45:19.597984 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.598680 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.600017 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.600472 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.600587 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.601101 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:45:19.601595 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:45:19.601741 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:19.601841 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 02:45:19.601924 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:45:19.610096 9986 raft_consensus.cc:1261] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67 [term 1 FOLLOWER]: Refusing update from remote peer a3ac2c59fea84533939ed58dbba629f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:45:19.610320 10397 raft_consensus.cc:1261] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff [term 1 FOLLOWER]: Refusing update from remote peer a3ac2c59fea84533939ed58dbba629f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:45:19.611012 10521 consensus_queue.cc:1035] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193413835bd042d0b8e50145424aeb67" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:45:19.611362 10514 consensus_queue.cc:1035] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "510046c57366489a99a8b300c0b567ff" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 35309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:45:19.622712 10524 mvcc.cc:205] Tried to move back new op lower bound from 7316099561913397248 to 7316099561738465280. Current Snapshot: MvccSnapshot[applied={T|T < 7316099561913397248}]
I20260808 02:45:19.646441 9763 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:45:19.646736 9763 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:45:19.650348 9763 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5579bd139948445b83a9c052d38173bd: Sending DeleteTablet for 3 replicas of tablet 91758e7c78aa4f16a81ad8b8b9549c61
I20260808 02:45:19.651551 9955 tablet_service.cc:1514] Processing DeleteTablet for tablet 91758e7c78aa4f16a81ad8b8b9549c61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:45:19 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:37446
I20260808 02:45:19.651928 10362 tablet_service.cc:1514] Processing DeleteTablet for tablet 91758e7c78aa4f16a81ad8b8b9549c61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:45:19 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:45402
I20260808 02:45:19.652671 10155 tablet_service.cc:1514] Processing DeleteTablet for tablet 91758e7c78aa4f16a81ad8b8b9549c61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:45:19 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:57092
I20260808 02:45:19.654945 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 9857
I20260808 02:45:19.666879 10537 tablet_replica.cc:330] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: stopping tablet replica
I20260808 02:45:19.667538 10537 raft_consensus.cc:2229] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:45:19.668236 10537 raft_consensus.cc:2258] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:45:19.669209 10538 tablet_replica.cc:330] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: stopping tablet replica
I20260808 02:45:19.669701 10538 raft_consensus.cc:2229] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:45:19.671140 10538 raft_consensus.cc:2258] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:45:19.673650 10538 ts_tablet_manager.cc:1905] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:45:19.674206 10537 ts_tablet_manager.cc:1905] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:45:19.675396 10539 tablet_replica.cc:330] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: stopping tablet replica
Aug 08 02:45:19 dist-test-slave-k5rr krb5kdc[9705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:45:19.678810 10539 raft_consensus.cc:2229] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:45:19.681491 10538 ts_tablet_manager.cc:1918] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:45:19.681741 10538 log.cc:1196] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/91758e7c78aa4f16a81ad8b8b9549c61
I20260808 02:45:19.682451 10537 ts_tablet_manager.cc:1918] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:45:19.682507 10538 ts_tablet_manager.cc:1939] T 91758e7c78aa4f16a81ad8b8b9549c61 P 510046c57366489a99a8b300c0b567ff: Deleting consensus metadata
I20260808 02:45:19.682731 10537 log.cc:1196] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/91758e7c78aa4f16a81ad8b8b9549c61
I20260808 02:45:19.683490 10537 ts_tablet_manager.cc:1939] T 91758e7c78aa4f16a81ad8b8b9549c61 P 193413835bd042d0b8e50145424aeb67: Deleting consensus metadata
I20260808 02:45:19.684657 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.656161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37460 (local address 127.25.137.193:34809)
0808 02:45:19.656350 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:19.656358 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:19.665599 (+ 9241us) server_negotiation.cc:407] Connection header received
0808 02:45:19.666093 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:19.666100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:19.666195 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:19.666273 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:19.668074 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.669262 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.671004 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.671599 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.671695 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.676797 (+ 5102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:19.676812 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:19.676830 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:45:19.676877 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:19.680818 (+ 3941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:19.681812 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:19.681821 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:19.681825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:19.682001 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:19.682539 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:19.682544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:19.682548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:19.683061 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:19.683397 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:19.684003 (+ 606us) server_negotiation.cc:299] Negotiation successful
0808 02:45:19.684241 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:45:19.685144 9744 catalog_manager.cc:4841] TS 510046c57366489a99a8b300c0b567ff (127.25.137.195:35309): tablet 91758e7c78aa4f16a81ad8b8b9549c61 (table test-table [id=379938c22dc541818a21b1943d03ae37]) successfully deleted
I20260808 02:45:19.685688 9745 catalog_manager.cc:4841] TS 193413835bd042d0b8e50145424aeb67 (127.25.137.193:34809): tablet 91758e7c78aa4f16a81ad8b8b9549c61 (table test-table [id=379938c22dc541818a21b1943d03ae37]) successfully deleted
I20260808 02:45:19.686021 10539 raft_consensus.cc:2258] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:45:19.686299 10043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:19.688660 10539 ts_tablet_manager.cc:1905] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:45:19.696642 10539 ts_tablet_manager.cc:1918] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:45:19.696866 10539 log.cc:1196] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/91758e7c78aa4f16a81ad8b8b9549c61
I20260808 02:45:19.697535 10539 ts_tablet_manager.cc:1939] T 91758e7c78aa4f16a81ad8b8b9549c61 P a3ac2c59fea84533939ed58dbba629f9: Deleting consensus metadata
I20260808 02:45:19.699574 9744 catalog_manager.cc:4841] TS a3ac2c59fea84533939ed58dbba629f9 (127.25.137.194:42783): tablet 91758e7c78aa4f16a81ad8b8b9549c61 (table test-table [id=379938c22dc541818a21b1943d03ae37]) successfully deleted
I20260808 02:45:19.776230 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 9857
I20260808 02:45:19.800294 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 10060
Aug 08 02:45:19 dist-test-slave-k5rr krb5kdc[9705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:45:19.815743 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.800996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57108 (local address 127.25.137.194:42783)
0808 02:45:19.801581 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:19.801587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:19.801604 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:45:19.801663 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:19.801667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:19.801751 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:19.801845 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:19.803054 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.803870 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.805083 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.805532 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.805637 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.809892 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:19.809904 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:19.809918 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:45:19.809958 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:19.812945 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:19.813814 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:19.813821 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:19.813823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:19.813956 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:19.814359 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:19.814362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:19.814364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:19.814708 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:19.814976 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:19.815315 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 02:45:19.815472 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435}
I20260808 02:45:19.816504 10243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:19.928925 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 10060
I20260808 02:45:19.953217 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 10258
Aug 08 02:45:19 dist-test-slave-k5rr krb5kdc[9705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:45:19.968516 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:45:19.954059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45406 (local address 127.25.137.195:35309)
0808 02:45:19.954280 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 02:45:19.954284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:45:19.954304 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:45:19.954498 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:45:19.954503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:45:19.954574 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:45:19.954665 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:45:19.956041 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.956742 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.958026 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:45:19.958417 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:45:19.958529 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:45:19.963108 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:45:19.963122 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:45:19.963138 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:45:19.963186 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:45:19.965808 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:19.966516 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:19.966524 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:19.966527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:19.966719 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:45:19.967156 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:45:19.967160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:45:19.967162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:45:19.967541 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:45:19.967770 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 02:45:19.968013 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 02:45:19.968178 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 02:45:19.969219 10457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:20.056396 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 10258
I20260808 02:45:20.078737 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 9723
I20260808 02:45:20.079885 9807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:45:20.189505 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 9723
2026-08-08T02:45:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4522 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:45:20 dist-test-slave-k5rr krb5kdc[10548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:45:20 dist-test-slave-k5rr krb5kdc[10548](info): set up 2 sockets
Aug 08 02:45:20 dist-test-slave-k5rr krb5kdc[10548](info): commencing operation
krb5kdc: starting...
W20260808 02:45:21.956688 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.717s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:45:22 dist-test-slave-k5rr krb5kdc[10548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:45:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:45:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/rangeradmin_127.25.137.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/rangeradmin_127.25.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/rangerlookup_127.25.137.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/rangerlookup_127.25.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/HTTP_127.25.137.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/HTTP_127.25.137.212@KRBTEST.COM.keytab.
I20260808 02:45:22.366504 26151 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOXTGli/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/postgres -l logfile start
2026-08-08 02:45:24.797 UTC [10599] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 02:45:24.797 UTC [10599] LOG: listening on IPv4 address "127.25.137.212", port 55747
2026-08-08 02:45:24.801 UTC [10599] LOG: listening on Unix socket "/tmp/.s.PGSQL.55747"
2026-08-08 02:45:24.823 UTC [10602] LOG: database system was shut down at 2026-08-08 02:45:23 UTC
2026-08-08 02:45:24.826 UTC [10599] LOG: database system is ready to accept connections
I20260808 02:45:25.905791 26151 mini_postgres.cc:96] Postgres bound to 55747
2026-08-08 02:45:25.932 UTC [10612] FATAL: database "slave" does not exist
127.25.137.212:55747 - accepting connections
I20260808 02:45:25.933050 26151 mini_ranger.cc:161] Starting Ranger...
I20260808 02:45:25.966338 26151 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 02:45:26.407619 26151 mini_ranger.cc:87] Created ranger Postgres database
I20260808 02:45:26.407769 26151 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-admin
2026-08-08 02:45:26,716 [I] DB FLAVOR :POSTGRES
2026-08-08 02:45:26,717 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:45:26,717 [I] Checking connection..
2026-08-08 02:45:26,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 02:45:26,974 [I] Checking connection passed.
2026-08-08 02:45:26,975 [I] --------- Verifying version history table ---------
2026-08-08 02:45:26,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:45:27,228 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:45:27,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:45:27,492 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:45:27,492 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 02:45:27,493 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 02:45:27.734 UTC [10710] WARNING: there is no transaction in progress
2026-08-08 02:45:27,743 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 02:45:27,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:45:28,003 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 02:45:28,003 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:45:28,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 02:45:28,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 02:45:28,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-k5rr.c.gcp-upstream.internal','N') ;"
2026-08-08 02:45:28,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:45:29,024 [I] Table x_portal_user does not exist in database ranger
2026-08-08 02:45:29,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:45:29,267 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 02:45:29,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:45:29,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:45:29,779 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 02:45:29,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 02:45:30.702 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.718 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.733 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.747 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.763 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.779 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.829 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.834 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.847 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.856 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.865 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.879 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.888 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.894 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.900 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:30.906 UTC [10868] WARNING: there is no transaction in progress
2026-08-08 02:45:31,478 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 02:45:31,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 02:45:31,737 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 02:45:31,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:45:32,014 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 02:45:32,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:45:32,276 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 02:45:32,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:45:32,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:45:32,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-k5rr.c.gcp-upstream.internal';"
2026-08-08 02:45:33,024 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 02:45:33,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 02:45:33,282 [I] DB_PATCHES have already been applied
I20260808 02:45:33.288378 26151 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOXTGli/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 02:45:33.288491 26151 mini_ranger.cc:193] Using host: 127.25.137.212
I20260808 02:45:33.310446 26151 mini_ranger.cc:227] Ranger admin URL: http://127.25.137.212:39955
Aug 08, 2026 2:45:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:45:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:45:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:45:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:45:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:45:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:45:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 02:45:34.598791 26151 mini_ranger.cc:160] Time spent starting Ranger: real 8.666s user 0.000s sys 0.069s
Aug 08, 2026 2:45:54 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:45:54 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 2:45:54 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:45:54 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:45:55 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 02:45:55.640575 26151 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/rangerkms_127.25.137.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/rangerkms_127.25.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.137.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.137.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.137.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/HTTP_127.25.137.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.137.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/HTTP_127.25.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 02:45:55 dist-test-slave-k5rr krb5kdc[10548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157155, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 02:45:55.838429 26151 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 02:45:55.868546 26151 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 02:45:56.628484 26151 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 02:45:56.628607 26151 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-kms
I20260808 02:45:56.628757 26151 mini_ranger_kms.cc:227] Using postgres at 127.25.137.212:55747
2026-08-08 02:45:57,289 [I] DB FLAVOR :POSTGRES
2026-08-08 02:45:57,289 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:45:57,289 [I] Checking connection
2026-08-08 02:45:57,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 02:45:57,556 [I] connection success
2026-08-08 02:45:57,556 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 02:45:57,556 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 02:45:57,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 02:45:57,800 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 02:45:57,800 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:45:57,800 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 02:45:57,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:55747/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 02:45:58,068 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 02:45:58.623684 26151 mini_ranger_kms.cc:310] Created kms service
I20260808 02:45:58.820640 26151 mini_ranger_kms.cc:326] Created kudu user
I20260808 02:45:58.865545 26151 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 02:45:58 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157155, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:45:59.266150 26151 mini_ranger_kms.cc:384] Added ranger policy
I20260808 02:45:59.266561 26151 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 02:45:59.266620 26151 mini_ranger_kms.cc:240] Using host: 127.25.137.212
I20260808 02:45:59.299182 26151 mini_ranger_kms.cc:276] Ranger KMS PID: 11144
I20260808 02:45:59.299372 26151 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.137.212:48637
Aug 08, 2026 2:46:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:46:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:46:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:46:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 02:46:01 dist-test-slave-k5rr krb5kdc[10548](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786157161, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.137.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:46:01.943796 26151 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.105s user 0.165s sys 0.414s
I20260808 02:46:01.944151 26151 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 02:46:01.944387 26151 mini_ranger_kms.cc:401] 127.25.137.212:48637/kms/v1/keys
Aug 08 02:46:02 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157161, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.137.212@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
Aug 08 02:46:02 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157161, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.137.212@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
Aug 08 02:46:04 dist-test-slave-k5rr krb5kdc[10548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:46:04.948838 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45171
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:45171
--ranger_config_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:48637/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:46:05.301322 11220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:46:05.301622 11220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:46:05.301859 11220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:46:05.309118 11220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:46:05.309221 11220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:46:05.309295 11220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:46:05.309366 11220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:46:05.309429 11220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:46:05.319949 11220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.137.212:48637/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:45171
--ranger_config_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:46:05.321352 11220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:46:05.322746 11220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:46:05.334669 11231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:05.334702 11230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:05.335695 11237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:46:05.335716 11220 server_base.cc:1034] running on GCE node
I20260808 02:46:05.336568 11220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:46:05.338065 11220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:46:05.339282 11220 hybrid_clock.cc:648] HybridClock initialized: now 1786157165339224 us; error 49 us; skew 500 ppm
I20260808 02:46:05.339624 11220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:46:05 dist-test-slave-k5rr krb5kdc[10548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157165, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:46:05.343073 11220 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:46:05.344818 11220 webserver.cc:460] Webserver started at http://127.25.137.254:34789/ using document root <none> and password file <none>
I20260808 02:46:05.345625 11220 fs_manager.cc:359] Metadata directory not provided
I20260808 02:46:05.345750 11220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:46:05.346068 11220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:46:05 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157165, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:05.407713 11220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "259065289e434403b1dd1b32bc4da324"
format_stamp: "Formatted at 2026-08-08 02:46:05 on dist-test-slave-k5rr"
server_key: "bbf381cc81287c871ad3b27ffb167aeb"
server_key_iv: "095eee8fa5840038c0c73fef5e49342f"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:05.408805 11220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "259065289e434403b1dd1b32bc4da324"
format_stamp: "Formatted at 2026-08-08 02:46:05 on dist-test-slave-k5rr"
server_key: "bbf381cc81287c871ad3b27ffb167aeb"
server_key_iv: "095eee8fa5840038c0c73fef5e49342f"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:05.415637 11220 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 02:46:05.439224 11252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:05.441193 11220 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.001s
I20260808 02:46:05.441396 11220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "259065289e434403b1dd1b32bc4da324"
format_stamp: "Formatted at 2026-08-08 02:46:05 on dist-test-slave-k5rr"
server_key: "bbf381cc81287c871ad3b27ffb167aeb"
server_key_iv: "095eee8fa5840038c0c73fef5e49342f"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:05.441665 11220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:46:05.463639 11220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:46:05.468757 11220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:46:05.469130 11220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:46:05.499506 11220 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:45171
I20260808 02:46:05.499512 11330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:45171 every 8 connection(s)
I20260808 02:46:05.501268 11220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:46:05.505208 11333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:46:05.505525 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 11220
I20260808 02:46:05.505734 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:46:05.506575 26151 external_mini_cluster.cc:1428] Setting key 91d9abe6ab0256ad30f99855d13c50c1
I20260808 02:46:05.520640 11333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324: Bootstrap starting.
Aug 08 02:46:05 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:46:05.524407 11333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324: Neither blocks nor log segments found. Creating new log.
I20260808 02:46:05.525775 11333 log.cc:824] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324: Log is configured to *not* fsync() on all Append() calls
I20260808 02:46:05.529537 11333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324: No bootstrap required, opened a new log
I20260808 02:46:05.533752 11338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:05.508707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.25.137.254:45171)
0808 02:46:05.509634 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:05.509649 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:05.509699 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:46:05.511890 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:05.511923 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:05.512449 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:05.512910 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:05.514517 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:05.516603 (+ 2086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:05.518436 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:05.519479 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:05.519662 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:05.524749 (+ 5087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:05.524762 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:05.524782 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:05.524856 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:05.528699 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:05.529675 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:05.529684 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:05.529693 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:05.529911 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:05.530362 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:05.530368 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:05.530371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:05.531053 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:05.531351 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:05.531929 (+ 578us) server_negotiation.cc:299] Negotiation successful
0808 02:46:05.532359 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":404,"threads_started":1}
I20260808 02:46:05.538964 11333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259065289e434403b1dd1b32bc4da324" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45171 } }
I20260808 02:46:05.539378 11333 raft_consensus.cc:374] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:46:05.539491 11333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 259065289e434403b1dd1b32bc4da324, State: Initialized, Role: FOLLOWER
I20260808 02:46:05.540014 11333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259065289e434403b1dd1b32bc4da324" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45171 } }
I20260808 02:46:05.540223 11333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:46:05.540354 11333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:46:05.540493 11333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:46:05.542708 11333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259065289e434403b1dd1b32bc4da324" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45171 } }
I20260808 02:46:05.543161 11333 leader_election.cc:302] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 259065289e434403b1dd1b32bc4da324; no voters:
I20260808 02:46:05.543715 11333 leader_election.cc:288] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:46:05.543883 11340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:46:05.544904 11340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [term 1 LEADER]: Becoming Leader. State: Replica: 259065289e434403b1dd1b32bc4da324, State: Running, Role: LEADER
I20260808 02:46:05.545593 11340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259065289e434403b1dd1b32bc4da324" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45171 } }
I20260808 02:46:05.545754 11333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:46:05.550678 11342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 259065289e434403b1dd1b32bc4da324. Latest consensus state: current_term: 1 leader_uuid: "259065289e434403b1dd1b32bc4da324" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259065289e434403b1dd1b32bc4da324" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45171 } } }
I20260808 02:46:05.551147 11342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [sys.catalog]: This master's current role is: LEADER
I20260808 02:46:05.550413 11341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "259065289e434403b1dd1b32bc4da324" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259065289e434403b1dd1b32bc4da324" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45171 } } }
I20260808 02:46:05.551949 11341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324 [sys.catalog]: This master's current role is: LEADER
I20260808 02:46:05.560901 11353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:46:05.569429 11353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:46:05.591511 11353 catalog_manager.cc:1347] Generated new cluster ID: 868aa1e2ab304334ab642ddace8272fd
I20260808 02:46:05.591642 11353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:46:05.601876 11333 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 02:46:05.623904 11353 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:46:05.626439 11353 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:46:05.644068 11353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324: Generated new TSK 0
I20260808 02:46:05.645416 11353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:46:06.444450 11398 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:46:06.557293 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:48637/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45171
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:46:06 dist-test-slave-k5rr krb5kdc[10548](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786157166, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 02:46:06.967437 11402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:46:06.967726 11402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:46:06.967902 11402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:46:06.975581 11402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:46:06.975705 11402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:46:06.975885 11402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:46:06.986361 11402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.137.212:48637/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:46:06.987694 11402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:46:06.988992 11402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:46:07.009049 11420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:07.008991 11419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:07.016000 11422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:46:07.018030 11402 server_base.cc:1034] running on GCE node
I20260808 02:46:07.018729 11402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:46:07.019820 11402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:46:07.021072 11402 hybrid_clock.cc:648] HybridClock initialized: now 1786157167021003 us; error 51 us; skew 500 ppm
I20260808 02:46:07.021425 11402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:46:07 dist-test-slave-k5rr krb5kdc[10548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157167, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:46:07.024653 11402 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:46:07.030195 11402 webserver.cc:460] Webserver started at http://127.25.137.193:33323/ using document root <none> and password file <none>
I20260808 02:46:07.030990 11402 fs_manager.cc:359] Metadata directory not provided
I20260808 02:46:07.031139 11402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:46:07.031538 11402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:46:07 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157167, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:07.052069 11402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "b5d65b3692cc415f97b0e966acc4a104"
format_stamp: "Formatted at 2026-08-08 02:46:07 on dist-test-slave-k5rr"
server_key: "804c7c41aa31fb54df2c89878135a07a"
server_key_iv: "c41fc58c973030bad612a2419f2b1c91"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:07.053041 11402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "b5d65b3692cc415f97b0e966acc4a104"
format_stamp: "Formatted at 2026-08-08 02:46:07 on dist-test-slave-k5rr"
server_key: "804c7c41aa31fb54df2c89878135a07a"
server_key_iv: "c41fc58c973030bad612a2419f2b1c91"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:07.060529 11402 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260808 02:46:07.094597 11431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:07.098249 11402 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.004s sys 0.008s
I20260808 02:46:07.098462 11402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "b5d65b3692cc415f97b0e966acc4a104"
format_stamp: "Formatted at 2026-08-08 02:46:07 on dist-test-slave-k5rr"
server_key: "804c7c41aa31fb54df2c89878135a07a"
server_key_iv: "c41fc58c973030bad612a2419f2b1c91"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:07.098745 11402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:46:07.137400 11402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:46:07.141713 11402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:46:07.142030 11402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:46:07.143083 11402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:46:07.144528 11402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:46:07.144620 11402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:07.144745 11402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:46:07.144812 11402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:07.239121 11402 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:35017
I20260808 02:46:07.239161 11609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:35017 every 8 connection(s)
I20260808 02:46:07.240850 11402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:46:07.241592 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 11402
I20260808 02:46:07.241819 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:46:07.242645 26151 external_mini_cluster.cc:1428] Setting key aa66566b801bd17ef506a3adab1f8a50
Aug 08 02:46:07 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157167, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
I20260808 02:46:07.322810 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:07.249827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:54641 (local address 127.25.137.254:45171)
0808 02:46:07.293460 (+ 43633us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:07.293468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:07.293503 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:46:07.295276 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:07.295283 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:07.295407 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:07.295534 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:07.298576 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:07.299749 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:07.307327 (+ 7578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:07.307997 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:07.308134 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:07.312892 (+ 4758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:07.312899 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:07.312904 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:07.312947 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:07.316485 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:07.319489 (+ 3004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:07.319496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:07.319500 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:07.319669 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:07.320264 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:07.320269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:07.320273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:07.320624 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:07.320751 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:07.322174 (+ 1423us) server_negotiation.cc:299] Negotiation successful
0808 02:46:07.322415 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43463,"thread_start_us":43669,"threads_started":1}
I20260808 02:46:07.325363 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:07.251168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45171 (local address 127.25.137.193:54641)
0808 02:46:07.282232 (+ 31064us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:07.282322 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:07.294707 (+ 12385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:07.296073 (+ 1366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:07.296087 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:07.297420 (+ 1333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:46:07.298340 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:07.298370 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:07.304605 (+ 6235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:07.304612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:07.307134 (+ 2522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:07.307155 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:07.308282 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:07.308292 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:07.308614 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:07.309572 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:46:07.309610 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:46:07.312608 (+ 2998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:46:07.316653 (+ 4045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:07.316664 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:07.316681 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:07.317310 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:07.319839 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:07.319844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:07.319848 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:07.320132 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:07.320858 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:46:07.320867 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:46:07.321341 (+ 474us) client_negotiation.cc:769] Sending connection context
0808 02:46:07.323296 (+ 1955us) client_negotiation.cc:240] Negotiation successful
0808 02:46:07.323877 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30640,"thread_start_us":34629,"threads_started":1}
I20260808 02:46:07.327952 11611 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45171
I20260808 02:46:07.328445 11611 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
I20260808 02:46:07.329519 11611 heartbeater.cc:511] Master 127.25.137.254:45171 requested a full tablet report, sending...
I20260808 02:46:07.332383 11276 ts_manager.cc:194] Registered new tserver with Master: b5d65b3692cc415f97b0e966acc4a104 (127.25.137.193:35017)
I20260808 02:46:07.334573 11276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:54641
Aug 08 02:46:07 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157166, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
W20260808 02:46:07.374811 26151 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.25.137.194: real 0.129s user 0.005s sys 0.021s
I20260808 02:46:07.375905 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:48637/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45171
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:46:07 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157166, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
W20260808 02:46:07.862771 11617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:46:07.863271 11617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:46:07.863489 11617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:46:07.875314 11617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:46:07.875464 11617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:46:07.875725 11617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:46:07.891691 11617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.137.212:48637/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:46:07.893602 11617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:46:07.895395 11617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:46:07.912535 11627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:07.914006 11628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:07.918342 11631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:46:07.919360 11617 server_base.cc:1034] running on GCE node
I20260808 02:46:07.920193 11617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:46:07.922297 11617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:46:07.923557 11617 hybrid_clock.cc:648] HybridClock initialized: now 1786157167923458 us; error 81 us; skew 500 ppm
I20260808 02:46:07.923961 11617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:46:07 dist-test-slave-k5rr krb5kdc[10548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157167, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:46:07.927482 11617 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:46:07.928972 11617 webserver.cc:460] Webserver started at http://127.25.137.194:40303/ using document root <none> and password file <none>
I20260808 02:46:07.929673 11617 fs_manager.cc:359] Metadata directory not provided
I20260808 02:46:07.929776 11617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:46:07.930038 11617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:46:07 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157167, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:07.946251 11617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "237ce883d5e746699b48011562c5c7c7"
format_stamp: "Formatted at 2026-08-08 02:46:07 on dist-test-slave-k5rr"
server_key: "cdef5fe399b2bee4798d0091bb1611f1"
server_key_iv: "2a4b2ad0a7660564fb1759d355cca611"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:07.947114 11617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "237ce883d5e746699b48011562c5c7c7"
format_stamp: "Formatted at 2026-08-08 02:46:07 on dist-test-slave-k5rr"
server_key: "cdef5fe399b2bee4798d0091bb1611f1"
server_key_iv: "2a4b2ad0a7660564fb1759d355cca611"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:07.953064 11617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:46:07.971282 11646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:07.972939 11617 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260808 02:46:07.973145 11617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "237ce883d5e746699b48011562c5c7c7"
format_stamp: "Formatted at 2026-08-08 02:46:07 on dist-test-slave-k5rr"
server_key: "cdef5fe399b2bee4798d0091bb1611f1"
server_key_iv: "2a4b2ad0a7660564fb1759d355cca611"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:07.973383 11617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:46:08.001997 11617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:46:08.007368 11617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:46:08.007717 11617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:46:08.008961 11617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:46:08.010928 11617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:46:08.011029 11617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:08.011158 11617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:46:08.011224 11617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:08.044955 11617 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:34581
I20260808 02:46:08.044968 11822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:34581 every 8 connection(s)
I20260808 02:46:08.046854 11617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:46:08.057258 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 11617
I20260808 02:46:08.057468 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:46:08.058040 26151 external_mini_cluster.cc:1428] Setting key e7c575c9b39894ce53a72abb913c3bdb
Aug 08 02:46:08 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157167, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:46:08.067919 11825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.049707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:53157 (local address 127.25.137.254:45171)
0808 02:46:08.050178 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:08.050182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:08.053076 (+ 2894us) server_negotiation.cc:407] Connection header received
0808 02:46:08.053626 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:08.053632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:08.053728 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:08.053836 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:08.055283 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.056078 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.057909 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.058400 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.058513 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.062216 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:08.062221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:08.062224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:08.062266 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:08.064752 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:08.065566 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:08.065570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:08.065572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:08.065695 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:08.066122 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:08.066126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:08.066128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:08.066420 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:08.066565 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:08.067323 (+ 758us) server_negotiation.cc:299] Negotiation successful
0808 02:46:08.067549 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":238,"threads_started":1}
I20260808 02:46:08.068976 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.050468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45171 (local address 127.25.137.194:53157)
0808 02:46:08.051408 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:08.051471 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:08.053363 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:08.054084 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:08.054092 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:08.054528 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:46:08.055108 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.055126 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.056303 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.056307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.057741 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.057756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.058621 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.058629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.058920 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.059664 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:46:08.059693 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:46:08.062004 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:46:08.064884 (+ 2880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:08.064892 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:08.064909 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:08.065451 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:08.065826 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:08.065830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:08.065834 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:08.066009 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:08.066547 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:46:08.066555 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:46:08.066948 (+ 393us) client_negotiation.cc:769] Sending connection context
0808 02:46:08.067186 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 02:46:08.067700 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":374,"threads_started":1}
I20260808 02:46:08.070393 11823 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45171
I20260808 02:46:08.070683 11823 heartbeater.cc:464] Registering TS with master...
I20260808 02:46:08.071446 11823 heartbeater.cc:511] Master 127.25.137.254:45171 requested a full tablet report, sending...
I20260808 02:46:08.072875 11276 ts_manager.cc:194] Registered new tserver with Master: 237ce883d5e746699b48011562c5c7c7 (127.25.137.194:34581)
I20260808 02:46:08.073819 11276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:53157
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:46:08.124632 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:48637/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45171
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:46:08.339349 11611 heartbeater.cc:503] Master 127.25.137.254:45171 was elected leader, sending a full tablet report...
W20260808 02:46:08.469841 11830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:46:08.470124 11830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:46:08.470288 11830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:46:08.477649 11830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:46:08.477762 11830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:46:08.477934 11830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:46:08.487702 11830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.137.212:48637/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:46:08.488952 11830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:46:08.490219 11830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:46:08.503582 11840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:08.503718 11839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:08.504294 11844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:46:08.504873 11830 server_base.cc:1034] running on GCE node
I20260808 02:46:08.505574 11830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:46:08.506487 11830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:46:08.507689 11830 hybrid_clock.cc:648] HybridClock initialized: now 1786157168507642 us; error 39 us; skew 500 ppm
I20260808 02:46:08.507999 11830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:46:08 dist-test-slave-k5rr krb5kdc[10548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157168, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:46:08.513396 11830 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:46:08.514871 11830 webserver.cc:460] Webserver started at http://127.25.137.195:43033/ using document root <none> and password file <none>
I20260808 02:46:08.515602 11830 fs_manager.cc:359] Metadata directory not provided
I20260808 02:46:08.515730 11830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:46:08.516026 11830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:46:08 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157168, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:08.532159 11830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "07c5e125633140b98e5bd3c06da4bf44"
format_stamp: "Formatted at 2026-08-08 02:46:08 on dist-test-slave-k5rr"
server_key: "e15f2e02320bfdb370f86bb60f4d32bb"
server_key_iv: "8f6fbe23180e3daa29d478d0b7e80e4b"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:08.532998 11830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "07c5e125633140b98e5bd3c06da4bf44"
format_stamp: "Formatted at 2026-08-08 02:46:08 on dist-test-slave-k5rr"
server_key: "e15f2e02320bfdb370f86bb60f4d32bb"
server_key_iv: "8f6fbe23180e3daa29d478d0b7e80e4b"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:08.538607 11830 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:46:08.555428 11856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:08.557274 11830 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:46:08.557492 11830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "07c5e125633140b98e5bd3c06da4bf44"
format_stamp: "Formatted at 2026-08-08 02:46:08 on dist-test-slave-k5rr"
server_key: "e15f2e02320bfdb370f86bb60f4d32bb"
server_key_iv: "8f6fbe23180e3daa29d478d0b7e80e4b"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:08.557781 11830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:46:08.578680 11830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:46:08.583827 11830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:46:08.584205 11830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:46:08.585451 11830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:46:08.587381 11830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:46:08.587488 11830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:08.587625 11830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:46:08.587702 11830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:08.623307 11830 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:34997
I20260808 02:46:08.623333 12040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:34997 every 8 connection(s)
I20260808 02:46:08.624680 11830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:46:08.626395 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 11830
I20260808 02:46:08.626570 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:46:08.627076 26151 external_mini_cluster.cc:1428] Setting key cb7504281821d7995ad2419c25671891
Aug 08 02:46:08 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157168, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:46:08.644853 12045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.627564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:50133 (local address 127.25.137.254:45171)
0808 02:46:08.628077 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:08.628081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:08.629083 (+ 1002us) server_negotiation.cc:407] Connection header received
0808 02:46:08.631377 (+ 2294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:08.631382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:08.631470 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:08.631579 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:08.633031 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.633784 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.635291 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.635715 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.635824 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.639185 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:08.639189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:08.639193 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:08.639225 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:08.641946 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:08.642641 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:08.642647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:08.642651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:08.642786 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:08.643182 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:08.643187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:08.643191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:08.643466 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:08.643594 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:08.644268 (+ 674us) server_negotiation.cc:299] Negotiation successful
0808 02:46:08.644495 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":266,"threads_started":1}
I20260808 02:46:08.645785 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.628117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45171 (local address 127.25.137.195:50133)
0808 02:46:08.628925 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:08.628962 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:08.631122 (+ 2160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:08.631841 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:08.631850 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:08.632283 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:46:08.632853 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.632870 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.633929 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.633933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.635154 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.635167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.635941 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.635949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.636184 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.636815 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:46:08.636837 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:46:08.638982 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:46:08.642091 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:08.642100 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:08.642114 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:08.642529 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:08.642891 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:08.642896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:08.642900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:08.643077 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:08.643592 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:46:08.643599 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:46:08.643917 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 02:46:08.644156 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 02:46:08.644659 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":350,"threads_started":1}
I20260808 02:46:08.647167 12041 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45171
I20260808 02:46:08.647446 12041 heartbeater.cc:464] Registering TS with master...
I20260808 02:46:08.648128 12041 heartbeater.cc:511] Master 127.25.137.254:45171 requested a full tablet report, sending...
I20260808 02:46:08.649580 11276 ts_manager.cc:194] Registered new tserver with Master: 07c5e125633140b98e5bd3c06da4bf44 (127.25.137.195:34997)
I20260808 02:46:08.650408 11276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:50133
I20260808 02:46:08.655290 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:46:08.672323 12045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.659749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48892 (local address 127.25.137.254:45171)
0808 02:46:08.659911 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:08.659917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:08.660357 (+ 440us) server_negotiation.cc:407] Connection header received
0808 02:46:08.660527 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:08.660531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:08.660605 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:08.660688 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:08.662240 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.662941 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.664318 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.664742 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.664851 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.667210 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:08.667216 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:08.667219 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:08.667252 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:08.669729 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:08.670428 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:08.670432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:08.670434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:08.670560 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:08.670923 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:08.670929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:08.670932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:08.671207 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:08.671461 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:08.671790 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 02:46:08.672011 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:46:08.676204 11276 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:46:08.683597 11276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:46:08.710630 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.697196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:35017 (local address 127.0.0.1:36360)
0808 02:46:08.699017 (+ 1821us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:08.699062 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:08.699294 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:08.700543 (+ 1249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:08.700553 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:08.700600 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:46:08.701210 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.701237 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.703708 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.703715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.706578 (+ 2863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.706600 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.709366 (+ 2766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.709374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.709650 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.709715 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 02:46:08.709879 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 02:46:08.710128 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1511,"thread_start_us":760,"threads_started":1}
I20260808 02:46:08.711966 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.697515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36360 (local address 127.25.137.193:35017)
0808 02:46:08.698544 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:08.698556 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:08.699519 (+ 963us) server_negotiation.cc:407] Connection header received
0808 02:46:08.699664 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:08.699674 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:08.699989 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:08.700204 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:46:08.701484 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.703496 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.706852 (+ 3356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.708649 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.708912 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.709162 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:08.711347 (+ 2185us) server_negotiation.cc:299] Negotiation successful
0808 02:46:08.711565 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":817,"thread_start_us":893,"threads_started":1}
I20260808 02:46:08.718854 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.698218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:34581 (local address 127.0.0.1:42302)
0808 02:46:08.710991 (+ 12773us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:08.711016 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:08.711145 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:08.712219 (+ 1074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:08.712224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:08.712267 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:46:08.712415 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.712424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.714245 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.714250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.716094 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.716115 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.718103 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.718110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.718298 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.718333 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:46:08.718429 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 02:46:08.718574 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12648,"thread_start_us":13701,"threads_started":1}
I20260808 02:46:08.719053 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.698294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42302 (local address 127.25.137.194:34581)
0808 02:46:08.711404 (+ 13110us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:08.711415 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:08.711450 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:46:08.711596 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:08.711605 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:08.711885 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:08.712341 (+ 456us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:46:08.712605 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.714063 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.716288 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.717921 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.718106 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.718299 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:08.718514 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 02:46:08.718687 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12920,"thread_start_us":12797,"threads_started":1}
I20260808 02:46:08.718470 11509 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8da692c14347898bb4f230a9d1a276 (DEFAULT_TABLE table=test-table [id=ca8ca16a74e04c72a171f4fee36c9bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:46:08.720188 11509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8da692c14347898bb4f230a9d1a276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:46:08.720777 12068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.698859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:34997 (local address 127.0.0.1:55746)
0808 02:46:08.711682 (+ 12823us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:08.711701 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:08.711829 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:08.713632 (+ 1803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:08.713637 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:08.713664 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:46:08.714113 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.714127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.715795 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.715801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.717916 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.717933 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.719981 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.719988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.720157 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.720193 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:46:08.720299 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 02:46:08.720455 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12692,"thread_start_us":14358,"threads_started":1}
I20260808 02:46:08.721602 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.699513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55746 (local address 127.25.137.195:34997)
0808 02:46:08.712982 (+ 13469us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:08.712988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:08.713050 (+ 62us) server_negotiation.cc:407] Connection header received
0808 02:46:08.713160 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:08.713169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:08.713437 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:08.713629 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:46:08.714382 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.715651 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.718160 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.719755 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.719969 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.720147 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:08.721042 (+ 895us) server_negotiation.cc:299] Negotiation successful
0808 02:46:08.721227 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13296,"thread_start_us":13139,"threads_started":1}
I20260808 02:46:08.722827 11718 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8da692c14347898bb4f230a9d1a276 (DEFAULT_TABLE table=test-table [id=ca8ca16a74e04c72a171f4fee36c9bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:46:08.724347 11718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8da692c14347898bb4f230a9d1a276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:46:08.725596 11939 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8da692c14347898bb4f230a9d1a276 (DEFAULT_TABLE table=test-table [id=ca8ca16a74e04c72a171f4fee36c9bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:46:08.727151 11939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8da692c14347898bb4f230a9d1a276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:46:08.732652 12076 tablet_bootstrap.cc:492] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: Bootstrap starting.
I20260808 02:46:08.735812 12076 tablet_bootstrap.cc:654] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: Neither blocks nor log segments found. Creating new log.
I20260808 02:46:08.737169 12076 log.cc:824] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: Log is configured to *not* fsync() on all Append() calls
I20260808 02:46:08.739282 12078 tablet_bootstrap.cc:492] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: Bootstrap starting.
I20260808 02:46:08.740233 12076 tablet_bootstrap.cc:492] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: No bootstrap required, opened a new log
I20260808 02:46:08.740522 12076 ts_tablet_manager.cc:1397] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 02:46:08.743018 12078 tablet_bootstrap.cc:654] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: Neither blocks nor log segments found. Creating new log.
I20260808 02:46:08.744403 12079 tablet_bootstrap.cc:492] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44: Bootstrap starting.
I20260808 02:46:08.744992 12078 log.cc:824] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:46:08.748086 12079 tablet_bootstrap.cc:654] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44: Neither blocks nor log segments found. Creating new log.
I20260808 02:46:08.748380 12078 tablet_bootstrap.cc:492] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: No bootstrap required, opened a new log
I20260808 02:46:08.748651 12078 ts_tablet_manager.cc:1397] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260808 02:46:08.749301 12079 log.cc:824] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44: Log is configured to *not* fsync() on all Append() calls
I20260808 02:46:08.750873 12076 raft_consensus.cc:348] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d65b3692cc415f97b0e966acc4a104" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 35017 } } peers { permanent_uuid: "07c5e125633140b98e5bd3c06da4bf44" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34997 } } peers { permanent_uuid: "237ce883d5e746699b48011562c5c7c7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34581 } }
I20260808 02:46:08.751329 12076 raft_consensus.cc:374] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:46:08.751439 12076 raft_consensus.cc:729] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5d65b3692cc415f97b0e966acc4a104, State: Initialized, Role: FOLLOWER
I20260808 02:46:08.752090 12076 consensus_queue.cc:260] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d65b3692cc415f97b0e966acc4a104" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 35017 } } peers { permanent_uuid: "07c5e125633140b98e5bd3c06da4bf44" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34997 } } peers { permanent_uuid: "237ce883d5e746699b48011562c5c7c7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34581 } }
I20260808 02:46:08.752401 12079 tablet_bootstrap.cc:492] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44: No bootstrap required, opened a new log
I20260808 02:46:08.752678 12079 ts_tablet_manager.cc:1397] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 02:46:08.760722 12078 raft_consensus.cc:348] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d65b3692cc415f97b0e966acc4a104" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 35017 } } peers { permanent_uuid: "07c5e125633140b98e5bd3c06da4bf44" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34997 } } peers { permanent_uuid: "237ce883d5e746699b48011562c5c7c7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34581 } }
I20260808 02:46:08.760850 12079 raft_consensus.cc:348] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d65b3692cc415f97b0e966acc4a104" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 35017 } } peers { permanent_uuid: "07c5e125633140b98e5bd3c06da4bf44" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34997 } } peers { permanent_uuid: "237ce883d5e746699b48011562c5c7c7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34581 } }
I20260808 02:46:08.761250 12078 raft_consensus.cc:374] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:46:08.761299 12079 raft_consensus.cc:374] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:46:08.761360 12078 raft_consensus.cc:729] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 237ce883d5e746699b48011562c5c7c7, State: Initialized, Role: FOLLOWER
I20260808 02:46:08.761415 12079 raft_consensus.cc:729] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07c5e125633140b98e5bd3c06da4bf44, State: Initialized, Role: FOLLOWER
I20260808 02:46:08.762154 12079 consensus_queue.cc:260] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d65b3692cc415f97b0e966acc4a104" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 35017 } } peers { permanent_uuid: "07c5e125633140b98e5bd3c06da4bf44" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34997 } } peers { permanent_uuid: "237ce883d5e746699b48011562c5c7c7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34581 } }
I20260808 02:46:08.762779 12076 ts_tablet_manager.cc:1428] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: Time spent starting tablet: real 0.022s user 0.012s sys 0.012s
I20260808 02:46:08.762727 12078 consensus_queue.cc:260] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d65b3692cc415f97b0e966acc4a104" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 35017 } } peers { permanent_uuid: "07c5e125633140b98e5bd3c06da4bf44" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34997 } } peers { permanent_uuid: "237ce883d5e746699b48011562c5c7c7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34581 } }
I20260808 02:46:08.763692 12041 heartbeater.cc:503] Master 127.25.137.254:45171 was elected leader, sending a full tablet report...
I20260808 02:46:08.764612 12079 ts_tablet_manager.cc:1428] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44: Time spent starting tablet: real 0.012s user 0.009s sys 0.005s
I20260808 02:46:08.766088 11823 heartbeater.cc:503] Master 127.25.137.254:45171 was elected leader, sending a full tablet report...
I20260808 02:46:08.766958 12078 ts_tablet_manager.cc:1428] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: Time spent starting tablet: real 0.018s user 0.017s sys 0.000s
I20260808 02:46:08.783417 12082 raft_consensus.cc:482] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:46:08.783686 12082 raft_consensus.cc:504] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d65b3692cc415f97b0e966acc4a104" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 35017 } } peers { permanent_uuid: "07c5e125633140b98e5bd3c06da4bf44" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34997 } } peers { permanent_uuid: "237ce883d5e746699b48011562c5c7c7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34581 } }
I20260808 02:46:08.785463 12082 leader_election.cc:288] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07c5e125633140b98e5bd3c06da4bf44 (127.25.137.195:34997), 237ce883d5e746699b48011562c5c7c7 (127.25.137.194:34581)
I20260808 02:46:08.794623 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.786085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:55563 (local address 127.25.137.194:34581)
0808 02:46:08.786311 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:08.786318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:08.787020 (+ 702us) server_negotiation.cc:407] Connection header received
0808 02:46:08.787196 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:08.787202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:08.787299 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:08.787391 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:46:08.788195 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.789422 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.792406 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.793654 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.793836 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.793913 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:08.794211 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 02:46:08.794355 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:46:08.794710 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.785973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:34581 (local address 127.25.137.193:55563)
0808 02:46:08.786926 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:08.786940 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:08.787057 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:08.787407 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:08.787412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:08.787445 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:46:08.787999 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.788019 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.789567 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.789571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.792210 (+ 2639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.792227 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.793894 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.793899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.794041 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.794067 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:46:08.794164 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 02:46:08.794304 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":164,"spinlock_wait_cycles":38016,"thread_start_us":573,"threads_started":1}
I20260808 02:46:08.794857 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.785822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:51565 (local address 127.25.137.195:34997)
0808 02:46:08.786044 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:08.786052 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:08.786995 (+ 943us) server_negotiation.cc:407] Connection header received
0808 02:46:08.787196 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:08.787202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:08.787286 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:08.787391 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:46:08.788200 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.789428 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.792407 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.793698 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.793912 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.793982 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:08.794483 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 02:46:08.794618 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:46:08.795583 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.785891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:34997 (local address 127.25.137.193:51565)
0808 02:46:08.786870 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:08.786890 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:08.787051 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:08.787407 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:08.787412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:08.787452 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:46:08.788061 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.788070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.789564 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.789569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.792210 (+ 2641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:08.792227 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.794095 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.794101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:08.794282 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.794316 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:46:08.795037 (+ 721us) client_negotiation.cc:240] Negotiation successful
0808 02:46:08.795199 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":170,"spinlock_wait_cycles":3968,"thread_start_us":541,"threads_started":1}
I20260808 02:46:08.796209 11754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8da692c14347898bb4f230a9d1a276" candidate_uuid: "b5d65b3692cc415f97b0e966acc4a104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "237ce883d5e746699b48011562c5c7c7" is_pre_election: true
I20260808 02:46:08.796823 11754 raft_consensus.cc:2454] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5d65b3692cc415f97b0e966acc4a104 in term 0.
I20260808 02:46:08.796690 11970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8da692c14347898bb4f230a9d1a276" candidate_uuid: "b5d65b3692cc415f97b0e966acc4a104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c5e125633140b98e5bd3c06da4bf44" is_pre_election: true
I20260808 02:46:08.797268 11970 raft_consensus.cc:2454] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5d65b3692cc415f97b0e966acc4a104 in term 0.
I20260808 02:46:08.797564 11432 leader_election.cc:302] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 237ce883d5e746699b48011562c5c7c7, b5d65b3692cc415f97b0e966acc4a104; no voters:
I20260808 02:46:08.797989 12082 raft_consensus.cc:2781] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:46:08.798117 12082 raft_consensus.cc:482] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:46:08.798216 12082 raft_consensus.cc:3037] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 0 FOLLOWER]: Advancing to term 1
W20260808 02:46:08.799263 11824 tablet.cc:2369] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:46:08.799908 11612 tablet.cc:2369] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:46:08.801048 12082 raft_consensus.cc:504] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d65b3692cc415f97b0e966acc4a104" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 35017 } } peers { permanent_uuid: "07c5e125633140b98e5bd3c06da4bf44" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34997 } } peers { permanent_uuid: "237ce883d5e746699b48011562c5c7c7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34581 } }
I20260808 02:46:08.801726 12082 leader_election.cc:288] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [CANDIDATE]: Term 1 election: Requested vote from peers 07c5e125633140b98e5bd3c06da4bf44 (127.25.137.195:34997), 237ce883d5e746699b48011562c5c7c7 (127.25.137.194:34581)
I20260808 02:46:08.802134 11970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8da692c14347898bb4f230a9d1a276" candidate_uuid: "b5d65b3692cc415f97b0e966acc4a104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c5e125633140b98e5bd3c06da4bf44"
I20260808 02:46:08.802284 11754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8da692c14347898bb4f230a9d1a276" candidate_uuid: "b5d65b3692cc415f97b0e966acc4a104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "237ce883d5e746699b48011562c5c7c7"
I20260808 02:46:08.802341 11970 raft_consensus.cc:3037] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:46:08.802472 11754 raft_consensus.cc:3037] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:46:08.804687 11754 raft_consensus.cc:2454] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5d65b3692cc415f97b0e966acc4a104 in term 1.
I20260808 02:46:08.804895 11970 raft_consensus.cc:2454] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5d65b3692cc415f97b0e966acc4a104 in term 1.
I20260808 02:46:08.805114 11432 leader_election.cc:302] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 237ce883d5e746699b48011562c5c7c7, b5d65b3692cc415f97b0e966acc4a104; no voters:
I20260808 02:46:08.805379 12082 raft_consensus.cc:2781] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:46:08.805852 12082 raft_consensus.cc:686] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 1 LEADER]: Becoming Leader. State: Replica: b5d65b3692cc415f97b0e966acc4a104, State: Running, Role: LEADER
I20260808 02:46:08.806288 12082 consensus_queue.cc:237] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d65b3692cc415f97b0e966acc4a104" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 35017 } } peers { permanent_uuid: "07c5e125633140b98e5bd3c06da4bf44" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34997 } } peers { permanent_uuid: "237ce883d5e746699b48011562c5c7c7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34581 } }
I20260808 02:46:08.810932 11276 catalog_manager.cc:5495] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5d65b3692cc415f97b0e966acc4a104 (127.25.137.193). New cstate: current_term: 1 leader_uuid: "b5d65b3692cc415f97b0e966acc4a104" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5d65b3692cc415f97b0e966acc4a104" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 35017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07c5e125633140b98e5bd3c06da4bf44" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "237ce883d5e746699b48011562c5c7c7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34581 } health_report { overall_health: UNKNOWN } } }
I20260808 02:46:08.826912 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.821592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.25.137.193:35017)
0808 02:46:08.821767 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:08.821802 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:08.821821 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:46:08.822008 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:08.822011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:08.822088 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:08.822207 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:46:08.822540 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.823338 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.824775 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.825222 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.825317 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.825915 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:46:08.826405 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:46:08.826517 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:08.826604 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 02:46:08.826668 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:46:08.835150 11970 raft_consensus.cc:1261] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44 [term 1 FOLLOWER]: Refusing update from remote peer b5d65b3692cc415f97b0e966acc4a104: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:46:08.835173 11754 raft_consensus.cc:1261] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7 [term 1 FOLLOWER]: Refusing update from remote peer b5d65b3692cc415f97b0e966acc4a104: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:46:08.836221 12090 consensus_queue.cc:1035] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [LEADER]: Connected to new peer: Peer: permanent_uuid: "237ce883d5e746699b48011562c5c7c7" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:46:08.836576 12082 consensus_queue.cc:1035] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07c5e125633140b98e5bd3c06da4bf44" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 34997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:46:08.846486 12093 mvcc.cc:205] Tried to move back new op lower bound from 7316099763538022400 to 7316099763433967616. Current Snapshot: MvccSnapshot[applied={T|T < 7316099763538022400}]
I20260808 02:46:08.848511 12095 mvcc.cc:205] Tried to move back new op lower bound from 7316099763538022400 to 7316099763433967616. Current Snapshot: MvccSnapshot[applied={T|T < 7316099763538022400}]
I20260808 02:46:08.861996 11273 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:46:08.862319 11273 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:46:08.866204 11273 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 259065289e434403b1dd1b32bc4da324: Sending DeleteTablet for 3 replicas of tablet 6e8da692c14347898bb4f230a9d1a276
I20260808 02:46:08.867401 11939 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8da692c14347898bb4f230a9d1a276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:46:08 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:55746
I20260808 02:46:08.867409 11509 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8da692c14347898bb4f230a9d1a276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:46:08 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:36360
I20260808 02:46:08.867489 11718 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8da692c14347898bb4f230a9d1a276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:46:08 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:42302
I20260808 02:46:08.870227 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 11402
I20260808 02:46:08.873520 12108 tablet_replica.cc:330] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: stopping tablet replica
I20260808 02:46:08.873986 12108 raft_consensus.cc:2229] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:46:08.874637 12108 raft_consensus.cc:2258] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:46:08.879789 12111 tablet_replica.cc:330] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: stopping tablet replica
I20260808 02:46:08.880264 12111 raft_consensus.cc:2229] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:46:08.880771 12111 raft_consensus.cc:2258] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:46:08.882994 12111 ts_tablet_manager.cc:1905] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:46:08.889961 12111 ts_tablet_manager.cc:1918] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:46:08.889948 12108 ts_tablet_manager.cc:1905] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:46:08.890115 12111 log.cc:1196] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/6e8da692c14347898bb4f230a9d1a276
I20260808 02:46:08.890600 12111 ts_tablet_manager.cc:1939] T 6e8da692c14347898bb4f230a9d1a276 P 237ce883d5e746699b48011562c5c7c7: Deleting consensus metadata
I20260808 02:46:08.892473 11253 catalog_manager.cc:4841] TS 237ce883d5e746699b48011562c5c7c7 (127.25.137.194:34581): tablet 6e8da692c14347898bb4f230a9d1a276 (table test-table [id=ca8ca16a74e04c72a171f4fee36c9bdf]) successfully deleted
I20260808 02:46:08.893287 12110 tablet_replica.cc:330] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44: stopping tablet replica
I20260808 02:46:08.893743 12110 raft_consensus.cc:2229] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:46:08.894295 12110 raft_consensus.cc:2258] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 08 02:46:08 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:46:08.896485 12110 ts_tablet_manager.cc:1905] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:46:08.897224 12108 ts_tablet_manager.cc:1918] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:46:08.897410 12108 log.cc:1196] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/6e8da692c14347898bb4f230a9d1a276
I20260808 02:46:08.897949 12108 ts_tablet_manager.cc:1939] T 6e8da692c14347898bb4f230a9d1a276 P b5d65b3692cc415f97b0e966acc4a104: Deleting consensus metadata
I20260808 02:46:08.899768 11254 catalog_manager.cc:4841] TS b5d65b3692cc415f97b0e966acc4a104 (127.25.137.193:35017): tablet 6e8da692c14347898bb4f230a9d1a276 (table test-table [id=ca8ca16a74e04c72a171f4fee36c9bdf]) successfully deleted
I20260808 02:46:08.902742 12110 ts_tablet_manager.cc:1918] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:46:08.902894 12110 log.cc:1196] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/6e8da692c14347898bb4f230a9d1a276
I20260808 02:46:08.903330 12110 ts_tablet_manager.cc:1939] T 6e8da692c14347898bb4f230a9d1a276 P 07c5e125633140b98e5bd3c06da4bf44: Deleting consensus metadata
I20260808 02:46:08.904882 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:08.871510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36382 (local address 127.25.137.193:35017)
0808 02:46:08.871709 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:08.871716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:08.882838 (+ 11122us) server_negotiation.cc:407] Connection header received
0808 02:46:08.883431 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:08.883438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:08.883532 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:08.883609 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:08.885594 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.886921 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.889116 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:08.889859 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:08.889963 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:08.896527 (+ 6564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:08.896541 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:08.896561 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:08.896608 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:08.901112 (+ 4504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:08.902026 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:08.902035 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:08.902039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:08.902207 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:08.902658 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:08.902663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:08.902666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:08.903189 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:08.903490 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:08.903987 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 02:46:08.904232 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:46:08.904985 11254 catalog_manager.cc:4841] TS 07c5e125633140b98e5bd3c06da4bf44 (127.25.137.195:34997): tablet 6e8da692c14347898bb4f230a9d1a276 (table test-table [id=ca8ca16a74e04c72a171f4fee36c9bdf]) successfully deleted
I20260808 02:46:08.905632 11599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:46:09.003662 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 11402
I20260808 02:46:09.029359 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 11617
Aug 08 02:46:09 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:46:09.044268 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:09.030142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42318 (local address 127.25.137.194:34581)
0808 02:46:09.030303 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:09.030307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:09.030326 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:46:09.030455 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:09.030460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:09.030523 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:09.030603 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:09.031872 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:09.032608 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:09.033923 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:09.034344 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:09.034437 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:09.038810 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:09.038827 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:09.038844 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:09.038886 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:09.041750 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:09.042399 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:09.042407 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:09.042409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:09.042539 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:09.042867 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:09.042870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:09.042872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:09.043275 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:09.043513 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:09.043763 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 02:46:09.043933 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:46:09.045158 11817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:46:09.134552 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 11617
I20260808 02:46:09.154003 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 11830
Aug 08 02:46:09 dist-test-slave-k5rr krb5kdc[10548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:46:09.169885 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:09.154868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55760 (local address 127.25.137.195:34997)
0808 02:46:09.155068 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:09.155072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:09.155093 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:46:09.155233 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:09.155237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:09.155304 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:09.155401 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:09.156774 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:09.157655 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:09.159050 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:09.159435 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:09.159536 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:09.163755 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:09.163770 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:09.163786 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:09.163855 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:09.166817 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:09.167549 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:09.167558 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:09.167560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:09.167708 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:09.168158 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:09.168163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:09.168165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:09.168578 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:09.168824 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:09.169319 (+ 495us) server_negotiation.cc:299] Negotiation successful
0808 02:46:09.169509 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:46:09.170574 12035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:46:09.260704 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 11830
I20260808 02:46:09.284478 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 11220
I20260808 02:46:09.285719 11324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:46:09.399402 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 11220
I20260808 02:46:09.431676 26151 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 02:46:09.776633 26151 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 02:46:09.776932 26151 mini_ranger.cc:66] Stopping Ranger...
I20260808 02:46:10.121884 26151 mini_ranger.cc:68] Stopped Ranger
2026-08-08 02:46:10.122 UTC [10599] LOG: received smart shutdown request
2026-08-08 02:46:10.125 UTC [10599] LOG: background worker "logical replication launcher" (PID 10608) exited with exit code 1
2026-08-08 02:46:10.125 UTC [10603] LOG: shutting down
2026-08-08 02:46:10.250 UTC [10599] LOG: database system is shut down
2026-08-08T02:46:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50163 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:46:10 dist-test-slave-k5rr krb5kdc[12133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:46:10 dist-test-slave-k5rr krb5kdc[12133](info): set up 2 sockets
Aug 08 02:46:10 dist-test-slave-k5rr krb5kdc[12133](info): commencing operation
krb5kdc: starting...
W20260808 02:46:11.582846 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.007s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:46:11 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:46:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:46:11Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/rangeradmin_127.25.137.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/rangeradmin_127.25.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/rangerlookup_127.25.137.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/rangerlookup_127.25.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/HTTP_127.25.137.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/HTTP_127.25.137.212@KRBTEST.COM.keytab.
I20260808 02:46:11.960126 26151 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOXTGli/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/postgres -l logfile start
2026-08-08 02:46:14.248 UTC [12177] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 02:46:14.248 UTC [12177] LOG: listening on IPv4 address "127.25.137.212", port 52489
2026-08-08 02:46:14.251 UTC [12177] LOG: listening on Unix socket "/tmp/.s.PGSQL.52489"
2026-08-08 02:46:14.274 UTC [12181] LOG: database system was shut down at 2026-08-08 02:46:13 UTC
2026-08-08 02:46:14.277 UTC [12177] LOG: database system is ready to accept connections
I20260808 02:46:15.390301 26151 mini_postgres.cc:96] Postgres bound to 52489
2026-08-08 02:46:15.413 UTC [12194] FATAL: database "slave" does not exist
127.25.137.212:52489 - accepting connections
I20260808 02:46:15.414194 26151 mini_ranger.cc:161] Starting Ranger...
I20260808 02:46:15.444165 26151 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 02:46:15.854653 26151 mini_ranger.cc:87] Created ranger Postgres database
I20260808 02:46:15.854795 26151 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-admin
2026-08-08 02:46:16,139 [I] DB FLAVOR :POSTGRES
2026-08-08 02:46:16,140 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:46:16,140 [I] Checking connection..
2026-08-08 02:46:16,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 02:46:16,399 [I] Checking connection passed.
2026-08-08 02:46:16,399 [I] --------- Verifying version history table ---------
2026-08-08 02:46:16,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:46:16,654 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:46:16,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:46:16,894 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:46:16,895 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 02:46:16,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 02:46:17.143 UTC [12306] WARNING: there is no transaction in progress
2026-08-08 02:46:17,152 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 02:46:17,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:46:17,396 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 02:46:17,396 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:46:17,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 02:46:17,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 02:46:17,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-k5rr.c.gcp-upstream.internal','N') ;"
2026-08-08 02:46:18,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:46:18,415 [I] Table x_portal_user does not exist in database ranger
2026-08-08 02:46:18,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:46:18,680 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 02:46:18,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:46:18,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:46:19,176 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 02:46:19,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 02:46:20.100 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.115 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.131 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.145 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.160 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.178 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.228 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.235 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.248 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.258 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.267 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.280 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.289 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.295 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.301 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20.307 UTC [12487] WARNING: there is no transaction in progress
2026-08-08 02:46:20,872 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 02:46:20,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 02:46:21,119 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 02:46:21,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:46:21,364 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 02:46:21,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:46:21,624 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 02:46:21,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:46:21,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:46:22,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-k5rr.c.gcp-upstream.internal';"
2026-08-08 02:46:22,396 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 02:46:22,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 02:46:22,655 [I] DB_PATCHES have already been applied
I20260808 02:46:22.661473 26151 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOXTGli/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 02:46:22.661561 26151 mini_ranger.cc:193] Using host: 127.25.137.212
I20260808 02:46:22.681993 26151 mini_ranger.cc:227] Ranger admin URL: http://127.25.137.212:39197
Aug 08, 2026 2:46:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:46:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:46:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:46:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:46:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:46:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:46:23 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 02:46:24.019454 26151 mini_ranger.cc:160] Time spent starting Ranger: real 8.605s user 0.001s sys 0.063s
Aug 08, 2026 2:46:44 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:46:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 2:46:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:46:44 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:46:45 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 02:46:45.837496 26151 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/rangerkms_127.25.137.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/rangerkms_127.25.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.137.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.137.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.137.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/HTTP_127.25.137.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.137.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/HTTP_127.25.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 02:46:46 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157206, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 02:46:46.035017 26151 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 02:46:46.060042 26151 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 02:46:46.807382 26151 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 02:46:46.807482 26151 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-kms
I20260808 02:46:46.807521 26151 mini_ranger_kms.cc:227] Using postgres at 127.25.137.212:52489
2026-08-08 02:46:47,165 [I] DB FLAVOR :POSTGRES
2026-08-08 02:46:47,165 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:46:47,165 [I] Checking connection
2026-08-08 02:46:47,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 02:46:47,421 [I] connection success
2026-08-08 02:46:47,421 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 02:46:47,421 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 02:46:47,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 02:46:47,677 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 02:46:47,677 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:46:47,677 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 02:46:47,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.137.212:52489/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 02:46:47,936 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 02:46:48.440414 26151 mini_ranger_kms.cc:310] Created kms service
I20260808 02:46:48.606815 26151 mini_ranger_kms.cc:326] Created kudu user
I20260808 02:46:48.649333 26151 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 02:46:48 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157206, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:49.037310 26151 mini_ranger_kms.cc:384] Added ranger policy
I20260808 02:46:49.037602 26151 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOXTGli/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOXTGli/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 02:46:49.037650 26151 mini_ranger_kms.cc:240] Using host: 127.25.137.212
I20260808 02:46:49.063740 26151 mini_ranger_kms.cc:276] Ranger KMS PID: 12786
I20260808 02:46:49.063889 26151 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.137.212:49567
Aug 08, 2026 2:46:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:46:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:46:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:46:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:46:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:46:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 02:46:51 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786157211, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.137.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 02:46:51 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157211, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.137.212@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:51.668186 26151 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.633s user 0.064s sys 0.333s
I20260808 02:46:51.668444 26151 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 02:46:51.668603 26151 mini_ranger_kms.cc:401] 127.25.137.212:49567/kms/v1/keys
Aug 08 02:46:52 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157211, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.137.212@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
Aug 08 02:46:54 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:46:54.394614 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:34715
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:34715
--ranger_config_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:49567/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:46:54.732522 12865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:46:54.732810 12865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:46:54.732939 12865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:46:54.740200 12865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:46:54.740306 12865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:46:54.740376 12865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:46:54.740453 12865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:46:54.740518 12865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:46:54.750521 12865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.137.212:49567/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:34715
--ranger_config_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:34715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:46:54.751809 12865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:46:54.753129 12865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:46:54.764261 12875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:54.764341 12876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:54.764333 12879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:46:54.765260 12865 server_base.cc:1034] running on GCE node
I20260808 02:46:54.765939 12865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:46:54.767205 12865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:46:54.768410 12865 hybrid_clock.cc:648] HybridClock initialized: now 1786157214768371 us; error 46 us; skew 500 ppm
I20260808 02:46:54.768728 12865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:46:54 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157214, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:46:54.771682 12865 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:46:54.773114 12865 webserver.cc:460] Webserver started at http://127.25.137.254:42991/ using document root <none> and password file <none>
I20260808 02:46:54.773686 12865 fs_manager.cc:359] Metadata directory not provided
I20260808 02:46:54.773782 12865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:46:54.774016 12865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:46:54 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157214, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:54.823441 12865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "df81646c8d6644eb9431cc626b4eae72"
format_stamp: "Formatted at 2026-08-08 02:46:54 on dist-test-slave-k5rr"
server_key: "46c553227b5c33272e4eece001dc65fc"
server_key_iv: "959e54f2ebfd9494f1d08df46b432506"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:54.824299 12865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "df81646c8d6644eb9431cc626b4eae72"
format_stamp: "Formatted at 2026-08-08 02:46:54 on dist-test-slave-k5rr"
server_key: "46c553227b5c33272e4eece001dc65fc"
server_key_iv: "959e54f2ebfd9494f1d08df46b432506"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:54.829521 12865 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 02:46:54.847276 12893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:54.848953 12865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:46:54.849185 12865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "df81646c8d6644eb9431cc626b4eae72"
format_stamp: "Formatted at 2026-08-08 02:46:54 on dist-test-slave-k5rr"
server_key: "46c553227b5c33272e4eece001dc65fc"
server_key_iv: "959e54f2ebfd9494f1d08df46b432506"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:54.849464 12865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:46:54.880330 12865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:46:54.885421 12865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:46:54.885784 12865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:46:54.904951 12865 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:34715
I20260808 02:46:54.904953 12967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:34715 every 8 connection(s)
I20260808 02:46:54.906740 12865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:46:54.910578 12968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:46:54.912765 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 12865
I20260808 02:46:54.912988 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:46:54.913901 26151 external_mini_cluster.cc:1428] Setting key 6cef79085176190d0464c6ca2bf64fd6
I20260808 02:46:54.925537 12968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72: Bootstrap starting.
I20260808 02:46:54.930083 12968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72: Neither blocks nor log segments found. Creating new log.
I20260808 02:46:54.931558 12968 log.cc:824] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72: Log is configured to *not* fsync() on all Append() calls
I20260808 02:46:54.934530 12968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72: No bootstrap required, opened a new log
I20260808 02:46:54.944028 12968 raft_consensus.cc:348] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } }
I20260808 02:46:54.944545 12968 raft_consensus.cc:374] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:46:54.944659 12968 raft_consensus.cc:729] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df81646c8d6644eb9431cc626b4eae72, State: Initialized, Role: FOLLOWER
I20260808 02:46:54.945406 12968 consensus_queue.cc:260] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } }
I20260808 02:46:54.945643 12968 raft_consensus.cc:388] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:46:54.945801 12968 raft_consensus.cc:482] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 08 02:46:54 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:46:54.945959 12968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:46:54.948683 12968 raft_consensus.cc:504] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } }
I20260808 02:46:54.949421 12968 leader_election.cc:302] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df81646c8d6644eb9431cc626b4eae72; no voters:
I20260808 02:46:54.950146 12968 leader_election.cc:288] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:46:54.950259 12974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:46:54.951464 12974 raft_consensus.cc:686] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 1 LEADER]: Becoming Leader. State: Replica: df81646c8d6644eb9431cc626b4eae72, State: Running, Role: LEADER
I20260808 02:46:54.952029 12974 consensus_queue.cc:237] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } }
I20260808 02:46:54.952574 12968 sys_catalog.cc:564] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:46:54.954789 12975 sys_catalog.cc:455] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df81646c8d6644eb9431cc626b4eae72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } } }
I20260808 02:46:54.955152 12975 sys_catalog.cc:458] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [sys.catalog]: This master's current role is: LEADER
I20260808 02:46:54.957868 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:54.916229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46782 (local address 127.25.137.254:34715)
0808 02:46:54.931122 (+ 14893us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:54.931139 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:54.931189 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:46:54.933893 (+ 2704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:54.933925 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:54.934412 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:54.934830 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:54.936612 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:54.938851 (+ 2239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:54.940798 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:54.942004 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:54.942243 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:54.947296 (+ 5053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:54.947308 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:54.947334 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:54.947392 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:54.951397 (+ 4005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:54.952425 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:54.952433 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:54.952441 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:54.952632 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:54.953153 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:54.953157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:54.953161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:54.953939 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:54.955634 (+ 1695us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:54.955940 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 02:46:54.956403 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14634,"thread_start_us":14366,"threads_started":1}
I20260808 02:46:54.961282 12976 sys_catalog.cc:455] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df81646c8d6644eb9431cc626b4eae72. Latest consensus state: current_term: 1 leader_uuid: "df81646c8d6644eb9431cc626b4eae72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } } }
I20260808 02:46:54.961632 12976 sys_catalog.cc:458] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [sys.catalog]: This master's current role is: LEADER
I20260808 02:46:54.962703 12981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:46:54.967936 12981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:46:54.974715 12968 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 02:46:54.989115 12981 catalog_manager.cc:1347] Generated new cluster ID: e7ddf0e329c540c2878becb8b144a458
I20260808 02:46:54.989413 12981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:46:55.002905 12981 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:46:55.004978 12981 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:46:55.025050 12981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72: Generated new TSK 0
I20260808 02:46:55.026242 12981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:46:55.791908 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:49567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34715
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:46:56 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786157216, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 02:46:56.261090 13047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:46:56.261546 13047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:46:56.261714 13047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:46:56.273378 13047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:46:56.273528 13047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:46:56.273766 13047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:46:56.289503 13047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.137.212:49567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:46:56.291567 13047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:46:56.293448 13047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:46:56.311522 13067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:56.312315 13060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:56.313647 13062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:46:56.314263 13047 server_base.cc:1034] running on GCE node
I20260808 02:46:56.315053 13047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:46:56.316366 13047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:46:56.317677 13047 hybrid_clock.cc:648] HybridClock initialized: now 1786157216317572 us; error 89 us; skew 500 ppm
I20260808 02:46:56.318074 13047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:46:56 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157216, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:46:56.322563 13047 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:46:56.334637 13047 webserver.cc:460] Webserver started at http://127.25.137.193:42023/ using document root <none> and password file <none>
I20260808 02:46:56.335434 13047 fs_manager.cc:359] Metadata directory not provided
I20260808 02:46:56.335568 13047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:46:56.335886 13047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:46:56 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157216, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:56.354532 13047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "4b23f65b36b8482da3b3ca89c5af5c42"
format_stamp: "Formatted at 2026-08-08 02:46:56 on dist-test-slave-k5rr"
server_key: "a58688affbe97b81d4bd344c38441fc1"
server_key_iv: "606a099bf7842aeaa9e59aa0410697df"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:56.355681 13047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "4b23f65b36b8482da3b3ca89c5af5c42"
format_stamp: "Formatted at 2026-08-08 02:46:56 on dist-test-slave-k5rr"
server_key: "a58688affbe97b81d4bd344c38441fc1"
server_key_iv: "606a099bf7842aeaa9e59aa0410697df"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:56.372447 13047 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.007s sys 0.008s
I20260808 02:46:56.397655 13080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:56.399577 13047 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.007s
I20260808 02:46:56.399770 13047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "4b23f65b36b8482da3b3ca89c5af5c42"
format_stamp: "Formatted at 2026-08-08 02:46:56 on dist-test-slave-k5rr"
server_key: "a58688affbe97b81d4bd344c38441fc1"
server_key_iv: "606a099bf7842aeaa9e59aa0410697df"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:56.400148 13047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:46:56.435302 13047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:46:56.439301 13047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:46:56.439601 13047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:46:56.440742 13047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:46:56.442377 13047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:46:56.442555 13047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 02:46:56.442849 13047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:46:56.442978 13047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:56.519573 13047 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:41297
I20260808 02:46:56.519606 13253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:41297 every 8 connection(s)
I20260808 02:46:56.521454 13047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:46:56.529363 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 13047
I20260808 02:46:56.529613 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:46:56.530479 26151 external_mini_cluster.cc:1428] Setting key 8faca285d1c351abfe971e66126e35eb
Aug 08 02:46:56 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157216, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
Aug 08 02:46:56 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157216, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
I20260808 02:46:56.575734 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:56.531647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:59987 (local address 127.25.137.254:34715)
0808 02:46:56.550981 (+ 19334us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:56.550988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:56.551011 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:46:56.551078 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:56.551083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:56.551188 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:56.551316 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:56.553520 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:56.554749 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:56.557253 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:56.557886 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:56.558023 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:56.566889 (+ 8866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:56.566896 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:56.566902 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:56.566966 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:56.570695 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:56.572309 (+ 1614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:56.572316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:56.572320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:56.572485 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:56.573280 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:56.573286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:56.573290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:56.573687 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:56.573852 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:56.575048 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0808 02:46:56.575290 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19179,"thread_start_us":19820,"threads_started":1}
I20260808 02:46:56.577046 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:56.532211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34715 (local address 127.25.137.193:59987)
0808 02:46:56.546545 (+ 14334us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:56.546619 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:56.549076 (+ 2457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:56.551641 (+ 2565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:56.551650 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:56.552264 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:46:56.553250 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:56.553281 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:56.554932 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:56.554938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:56.557034 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:56.557064 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:56.558364 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:56.558371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:56.558668 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:56.559484 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:46:56.559523 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:46:56.566395 (+ 6872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:46:56.570872 (+ 4477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:56.570882 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:56.570902 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:56.571482 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:56.572628 (+ 1146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:56.572632 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:56.572635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:56.572912 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:56.573834 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:46:56.573843 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:46:56.574283 (+ 440us) client_negotiation.cc:769] Sending connection context
0808 02:46:56.574662 (+ 379us) client_negotiation.cc:240] Negotiation successful
0808 02:46:56.575348 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13843,"thread_start_us":15791,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
I20260808 02:46:56.582649 13255 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34715
I20260808 02:46:56.583027 13255 heartbeater.cc:464] Registering TS with master...
I20260808 02:46:56.584005 13255 heartbeater.cc:511] Master 127.25.137.254:34715 requested a full tablet report, sending...
I20260808 02:46:56.586624 12919 ts_manager.cc:194] Registered new tserver with Master: 4b23f65b36b8482da3b3ca89c5af5c42 (127.25.137.193:41297)
I20260808 02:46:56.588706 12919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:59987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:46:56.621165 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:49567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34715
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:46:56 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157216, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
W20260808 02:46:57.068984 13261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:46:57.069548 13261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:46:57.069763 13261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:46:57.081576 13261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:46:57.081792 13261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:46:57.082032 13261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:46:57.097663 13261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.137.212:49567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:46:57.099664 13261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:46:57.101643 13261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:46:57.137183 13276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:57.142079 13280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:57.142289 13275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:46:57.142977 13261 server_base.cc:1034] running on GCE node
I20260808 02:46:57.143950 13261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:46:57.150542 13261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:46:57.151907 13261 hybrid_clock.cc:648] HybridClock initialized: now 1786157217151796 us; error 109 us; skew 500 ppm
I20260808 02:46:57.152379 13261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:46:57 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157217, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:46:57.156992 13261 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:46:57.173435 13261 webserver.cc:460] Webserver started at http://127.25.137.194:44343/ using document root <none> and password file <none>
I20260808 02:46:57.174239 13261 fs_manager.cc:359] Metadata directory not provided
I20260808 02:46:57.174466 13261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:46:57.174803 13261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:46:57 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157217, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:57.194341 13261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "698aa7593bc4449ca84c2cf05152710e"
format_stamp: "Formatted at 2026-08-08 02:46:57 on dist-test-slave-k5rr"
server_key: "4a3493c99595a451dcd4cb2910014f46"
server_key_iv: "dbc0417e056bd929cdda98cc8f8a02ef"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:57.195705 13261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "698aa7593bc4449ca84c2cf05152710e"
format_stamp: "Formatted at 2026-08-08 02:46:57 on dist-test-slave-k5rr"
server_key: "4a3493c99595a451dcd4cb2910014f46"
server_key_iv: "dbc0417e056bd929cdda98cc8f8a02ef"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:57.213217 13261 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.012s sys 0.003s
I20260808 02:46:57.230594 13298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:57.232419 13261 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:46:57.232599 13261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "698aa7593bc4449ca84c2cf05152710e"
format_stamp: "Formatted at 2026-08-08 02:46:57 on dist-test-slave-k5rr"
server_key: "4a3493c99595a451dcd4cb2910014f46"
server_key_iv: "dbc0417e056bd929cdda98cc8f8a02ef"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:57.232867 13261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:46:57.259743 13261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:46:57.263793 13261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:46:57.264081 13261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:46:57.265107 13261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:46:57.266733 13261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:46:57.266886 13261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 02:46:57.267009 13261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:46:57.267063 13261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:57.301852 13261 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:46211
I20260808 02:46:57.301855 13475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:46211 every 8 connection(s)
I20260808 02:46:57.303217 13261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:46:57.306813 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 13261
I20260808 02:46:57.306989 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:46:57.307605 26151 external_mini_cluster.cc:1428] Setting key 601eb9e3bfbf8e7bf6fee1033a2b656c
Aug 08 02:46:57 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157217, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:46:57.323992 13478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:57.306113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:48109 (local address 127.25.137.254:34715)
0808 02:46:57.306608 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:57.306614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:57.307677 (+ 1063us) server_negotiation.cc:407] Connection header received
0808 02:46:57.310172 (+ 2495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:57.310177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:57.310278 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:57.310343 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:57.311993 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:57.312705 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:57.314282 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:57.314681 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:57.314803 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:57.318303 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:57.318308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:57.318312 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:57.318343 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:57.320840 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:57.321562 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:57.321567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:57.321569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:57.321689 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:57.322165 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:57.322168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:57.322170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:57.322556 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:57.322705 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:57.323541 (+ 836us) server_negotiation.cc:299] Negotiation successful
0808 02:46:57.323722 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":2990,"threads_started":1}
I20260808 02:46:57.325340 13479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:57.306681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34715 (local address 127.25.137.194:48109)
0808 02:46:57.307518 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:57.307563 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:57.309851 (+ 2288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:57.310747 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:57.310755 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:57.311181 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:46:57.311833 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:57.311851 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:57.312866 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:57.312872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:57.314148 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:57.314161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:57.314902 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:57.314910 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:57.315142 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:57.315769 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:46:57.315794 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:46:57.318084 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:46:57.320970 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:57.320978 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:57.320994 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:57.321440 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:57.321799 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:57.321804 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:57.321808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:57.322050 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:57.322708 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:46:57.322717 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:46:57.323134 (+ 417us) client_negotiation.cc:769] Sending connection context
0808 02:46:57.323432 (+ 298us) client_negotiation.cc:240] Negotiation successful
0808 02:46:57.323959 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":321,"threads_started":1}
I20260808 02:46:57.326934 13476 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34715
I20260808 02:46:57.327272 13476 heartbeater.cc:464] Registering TS with master...
I20260808 02:46:57.328128 13476 heartbeater.cc:511] Master 127.25.137.254:34715 requested a full tablet report, sending...
I20260808 02:46:57.329814 12919 ts_manager.cc:194] Registered new tserver with Master: 698aa7593bc4449ca84c2cf05152710e (127.25.137.194:46211)
I20260808 02:46:57.330828 12919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:48109
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:46:57.378659 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:49567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34715
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:46:57.592437 13255 heartbeater.cc:503] Master 127.25.137.254:34715 was elected leader, sending a full tablet report...
W20260808 02:46:57.732724 13483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:46:57.733026 13483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:46:57.733145 13483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:46:57.740532 13483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:46:57.740643 13483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:46:57.740761 13483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:46:57.750631 13483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.137.212:49567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:34715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:46:57.751976 13483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:46:57.753345 13483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:46:57.766083 13489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:57.766145 13492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:57.766055 13495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:46:57.766973 13483 server_base.cc:1034] running on GCE node
I20260808 02:46:57.767537 13483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:46:57.768481 13483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:46:57.769685 13483 hybrid_clock.cc:648] HybridClock initialized: now 1786157217769635 us; error 31 us; skew 500 ppm
I20260808 02:46:57.770002 13483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:46:57 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157217, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:46:57.773243 13483 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:46:57.774653 13483 webserver.cc:460] Webserver started at http://127.25.137.195:38377/ using document root <none> and password file <none>
I20260808 02:46:57.775306 13483 fs_manager.cc:359] Metadata directory not provided
I20260808 02:46:57.775429 13483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:46:57.775718 13483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:46:57 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157217, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:57.789657 13483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "7815b26804394592956d84b863992c3d"
format_stamp: "Formatted at 2026-08-08 02:46:57 on dist-test-slave-k5rr"
server_key: "9a2efbc8969a24a5fad70110dcbf71cb"
server_key_iv: "257333fe544e41a12c7c4469398d6d7a"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:57.790526 13483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "7815b26804394592956d84b863992c3d"
format_stamp: "Formatted at 2026-08-08 02:46:57 on dist-test-slave-k5rr"
server_key: "9a2efbc8969a24a5fad70110dcbf71cb"
server_key_iv: "257333fe544e41a12c7c4469398d6d7a"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:57.796736 13483 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260808 02:46:57.812624 13510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:57.814198 13483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:46:57.814369 13483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "7815b26804394592956d84b863992c3d"
format_stamp: "Formatted at 2026-08-08 02:46:57 on dist-test-slave-k5rr"
server_key: "9a2efbc8969a24a5fad70110dcbf71cb"
server_key_iv: "257333fe544e41a12c7c4469398d6d7a"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:57.814592 13483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:46:57.838505 13483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:46:57.842136 13483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:46:57.842430 13483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:46:57.843379 13483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:46:57.844728 13483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:46:57.844818 13483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:57.844952 13483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:46:57.845022 13483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:57.877151 13483 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:39741
I20260808 02:46:57.877209 13705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:39741 every 8 connection(s)
I20260808 02:46:57.878460 13483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:46:57.882143 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 13483
I20260808 02:46:57.882308 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:46:57.882890 26151 external_mini_cluster.cc:1428] Setting key b004d1e2bcb00e8fd0fd2b3af6955be1
Aug 08 02:46:57 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157217, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:46:57.898473 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:57.881124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:50767 (local address 127.25.137.254:34715)
0808 02:46:57.881627 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:57.881632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:57.882710 (+ 1078us) server_negotiation.cc:407] Connection header received
0808 02:46:57.884691 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:57.884695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:57.884784 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:57.884947 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:57.886391 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:57.887170 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:57.888708 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:57.889298 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:57.889418 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:57.892674 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:57.892679 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:57.892682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:57.892714 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:57.895389 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:57.896092 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:57.896098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:57.896100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:57.896229 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:57.896740 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:57.896745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:57.896749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:57.897102 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:57.897235 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:57.897894 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 02:46:57.898117 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":244,"threads_started":1}
I20260808 02:46:57.899358 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:57.881719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34715 (local address 127.25.137.195:50767)
0808 02:46:57.882544 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:57.882588 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:57.884449 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:57.885167 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:57.885175 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:57.885654 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:46:57.886233 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:57.886250 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:57.887321 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:57.887325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:57.888555 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:57.888568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:57.889497 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:57.889502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:57.889705 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:57.890368 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:46:57.890396 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:46:57.892473 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:46:57.895530 (+ 3057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:57.895536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:57.895548 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:57.895970 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:57.896357 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:57.896360 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:57.896363 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:57.896633 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:57.897226 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:46:57.897233 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:46:57.897545 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 02:46:57.897784 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 02:46:57.898256 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":338,"threads_started":1}
I20260808 02:46:57.900557 13706 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34715
I20260808 02:46:57.900854 13706 heartbeater.cc:464] Registering TS with master...
I20260808 02:46:57.901595 13706 heartbeater.cc:511] Master 127.25.137.254:34715 requested a full tablet report, sending...
I20260808 02:46:57.902981 12919 ts_manager.cc:194] Registered new tserver with Master: 7815b26804394592956d84b863992c3d (127.25.137.195:39741)
I20260808 02:46:57.903790 12919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:50767
I20260808 02:46:57.910075 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:46:57.911978 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 13047
Aug 08 02:46:57 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:46:57.928210 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:57.913066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52336 (local address 127.25.137.193:41297)
0808 02:46:57.913617 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:57.913623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:57.913648 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:46:57.913702 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:57.913708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:57.913904 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:57.914022 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:57.915418 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:57.916485 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:57.918052 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:57.918588 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:57.918699 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:57.922836 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:57.922848 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:57.922863 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:57.922900 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:57.925590 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:57.926323 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:57.926330 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:57.926332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:57.926486 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:57.926776 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:57.926779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:57.926782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:57.927113 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:57.927338 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:57.927622 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 02:46:57.927879 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":256,"threads_started":1}
I20260808 02:46:57.929450 13247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:46:58.014508 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 13047
I20260808 02:46:58.035023 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 13261
Aug 08 02:46:58 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:46:58.050923 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:58.035945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51042 (local address 127.25.137.194:46211)
0808 02:46:58.036476 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:58.036483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:58.036508 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:46:58.036570 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:58.036577 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:58.036795 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:58.036937 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:58.038295 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:58.039359 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:58.040829 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:58.041459 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:58.041560 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:58.045740 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:58.045751 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:58.045765 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:58.045800 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:58.048453 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:58.049108 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:58.049114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:58.049117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:58.049247 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:58.049564 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:58.049567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:58.049569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:58.049889 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:58.050140 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:58.050376 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 02:46:58.050576 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":266,"threads_started":1}
I20260808 02:46:58.052217 13463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:46:58.134362 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 13261
I20260808 02:46:58.161119 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 13483
Aug 08 02:46:58 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:46:58.178069 13724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:58.162021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.25.137.195:39741)
0808 02:46:58.162633 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:58.162641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:58.162669 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:46:58.162731 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:58.162738 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:58.162950 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:58.163092 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:58.164476 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:58.165672 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:58.166988 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:58.167569 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:58.167677 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:58.172326 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:58.172340 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:58.172356 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:58.172394 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:58.175358 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:58.176010 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:58.176017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:58.176019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:58.176150 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:58.176475 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:58.176477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:58.176479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:58.176833 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:58.177146 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:58.177366 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 02:46:58.177570 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":290,"threads_started":1}
I20260808 02:46:58.179405 13700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:46:58.262454 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 13483
I20260808 02:46:58.285640 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 12865
I20260808 02:46:58.287002 12963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:46:58.396066 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 12865
I20260808 02:46:58.428797 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:34715
--webserver_interface=127.25.137.254
--webserver_port=42991
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:34715
--ranger_config_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:49567/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.137.212:49567/kms with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:46:58.783396 13728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:46:58.783720 13728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:46:58.783864 13728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:46:58.791093 13728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:46:58.791216 13728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:46:58.791298 13728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:46:58.791378 13728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:46:58.791452 13728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:46:58.801446 13728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.137.212:49567/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:34715
--ranger_config_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:34715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=42991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:46:58.802711 13728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:46:58.804042 13728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:46:58.815577 13738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:58.815605 13739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:58.815595 13741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:46:58.817025 13728 server_base.cc:1034] running on GCE node
I20260808 02:46:58.817834 13728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:46:58.819278 13728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:46:58.820504 13728 hybrid_clock.cc:648] HybridClock initialized: now 1786157218820448 us; error 54 us; skew 500 ppm
I20260808 02:46:58.820840 13728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:46:58 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157218, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:46:58.826892 13728 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:46:58.828464 13728 webserver.cc:460] Webserver started at http://127.25.137.254:42991/ using document root <none> and password file <none>
I20260808 02:46:58.829162 13728 fs_manager.cc:359] Metadata directory not provided
I20260808 02:46:58.829293 13728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:46:58 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157218, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:58.905054 13728 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 02:46:58.908100 13750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:58.909494 13728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 02:46:58.909665 13728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "df81646c8d6644eb9431cc626b4eae72"
format_stamp: "Formatted at 2026-08-08 02:46:54 on dist-test-slave-k5rr"
server_key: "46c553227b5c33272e4eece001dc65fc"
server_key_iv: "959e54f2ebfd9494f1d08df46b432506"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:58.910629 13728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:46:58.939458 13728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:46:58.943251 13728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:46:58.943532 13728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:46:58.962129 13728 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:34715
I20260808 02:46:58.962157 13828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:34715 every 8 connection(s)
I20260808 02:46:58.963585 13728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:46:58.967748 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 13728
I20260808 02:46:58.968626 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:49567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:41297
--local_ip_for_outbound_sockets=127.25.137.193
--tserver_master_addrs=127.25.137.254:34715
--webserver_port=42023
--webserver_interface=127.25.137.193
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.137.212:49567/kms with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:46:58.970726 13829 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:46:58.974153 13829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72: Bootstrap starting.
I20260808 02:46:58.993872 13829 log.cc:824] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72: Log is configured to *not* fsync() on all Append() calls
I20260808 02:46:59.003958 13829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:46:59.004703 13829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72: Bootstrap complete.
I20260808 02:46:59.014815 13829 raft_consensus.cc:348] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } }
I20260808 02:46:59.015767 13829 raft_consensus.cc:729] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: df81646c8d6644eb9431cc626b4eae72, State: Initialized, Role: FOLLOWER
I20260808 02:46:59.016530 13829 consensus_queue.cc:260] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } }
I20260808 02:46:59.016781 13829 raft_consensus.cc:388] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:46:59.016912 13829 raft_consensus.cc:482] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:46:59.017117 13829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:46:59.021131 13829 raft_consensus.cc:504] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } }
I20260808 02:46:59.021624 13829 leader_election.cc:302] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df81646c8d6644eb9431cc626b4eae72; no voters:
I20260808 02:46:59.022421 13829 leader_election.cc:288] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:46:59.022574 13839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:46:59.023662 13839 raft_consensus.cc:686] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [term 2 LEADER]: Becoming Leader. State: Replica: df81646c8d6644eb9431cc626b4eae72, State: Running, Role: LEADER
I20260808 02:46:59.024300 13839 consensus_queue.cc:237] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } }
I20260808 02:46:59.024644 13829 sys_catalog.cc:564] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:46:59.026929 13841 sys_catalog.cc:455] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "df81646c8d6644eb9431cc626b4eae72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } } }
I20260808 02:46:59.026960 13843 sys_catalog.cc:455] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df81646c8d6644eb9431cc626b4eae72. Latest consensus state: current_term: 2 leader_uuid: "df81646c8d6644eb9431cc626b4eae72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df81646c8d6644eb9431cc626b4eae72" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34715 } } }
I20260808 02:46:59.027384 13841 sys_catalog.cc:458] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [sys.catalog]: This master's current role is: LEADER
I20260808 02:46:59.027392 13843 sys_catalog.cc:458] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72 [sys.catalog]: This master's current role is: LEADER
I20260808 02:46:59.039408 13851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:46:59.049829 13851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:46:59.051302 13851 catalog_manager.cc:1259] Loaded cluster ID: e7ddf0e329c540c2878becb8b144a458
I20260808 02:46:59.051510 13851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:46:59.055608 13851 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:46:59.057127 13851 catalog_manager.cc:5879] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72: Loaded TSK: 0
I20260808 02:46:59.058904 13851 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:46:59.062628 13829 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260808 02:46:59.377681 13836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:46:59.378192 13836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:46:59.378420 13836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:46:59.390056 13836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:46:59.390293 13836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:46:59.390555 13836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:46:59.405445 13836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.137.212:49567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:41297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=42023
--tserver_master_addrs=127.25.137.254:34715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:46:59.407300 13836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:46:59.409200 13836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:46:59.434053 13888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:59.434018 13889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:46:59.445756 13891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:46:59.448021 13836 server_base.cc:1034] running on GCE node
I20260808 02:46:59.449103 13836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:46:59.457656 13836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:46:59.458993 13836 hybrid_clock.cc:648] HybridClock initialized: now 1786157219458922 us; error 66 us; skew 500 ppm
I20260808 02:46:59.459461 13836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:46:59 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157219, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:46:59.463135 13836 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:46:59.469080 13836 webserver.cc:460] Webserver started at http://127.25.137.193:42023/ using document root <none> and password file <none>
I20260808 02:46:59.469926 13836 fs_manager.cc:359] Metadata directory not provided
I20260808 02:46:59.470078 13836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:46:59 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157219, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:46:59.518919 13836 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.004s sys 0.000s
I20260808 02:46:59.523046 13902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:59.524804 13836 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 02:46:59.525228 13836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "4b23f65b36b8482da3b3ca89c5af5c42"
format_stamp: "Formatted at 2026-08-08 02:46:56 on dist-test-slave-k5rr"
server_key: "a58688affbe97b81d4bd344c38441fc1"
server_key_iv: "606a099bf7842aeaa9e59aa0410697df"
server_key_version: "kuduclusterkey@0"
I20260808 02:46:59.526628 13836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:46:59.590847 13836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:46:59.596195 13836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:46:59.596536 13836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:46:59.613498 13836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:46:59.618538 13836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:46:59.618677 13836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:59.618809 13836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:46:59.618892 13836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:46:59.696437 13836 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:41297
I20260808 02:46:59.696481 14076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:41297 every 8 connection(s)
I20260808 02:46:59.698661 13836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:46:59.706907 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 13836
I20260808 02:46:59.707723 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:49567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:46211
--local_ip_for_outbound_sockets=127.25.137.194
--tserver_master_addrs=127.25.137.254:34715
--webserver_port=44343
--webserver_interface=127.25.137.194
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.137.212:49567/kms with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:46:59 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157219, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:46:59.736975 14081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:59.707434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34715 (local address 127.25.137.193:36855)
0808 02:46:59.708961 (+ 1527us) negotiation.cc:107] Waiting for socket to connect
0808 02:46:59.709031 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 02:46:59.713639 (+ 4608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:46:59.716406 (+ 2767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:46:59.716418 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:46:59.717119 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:46:59.717907 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:59.717933 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:59.720129 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:59.720134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:59.722224 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:46:59.722256 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:46:59.723621 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:46:59.723630 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:46:59.723938 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:59.724679 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:46:59.724708 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:46:59.727299 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:46:59.731504 (+ 4205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:59.731513 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:59.731529 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:59.732174 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:59.732782 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:46:59.732787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:46:59.732790 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:46:59.733076 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:46:59.734215 (+ 1139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:46:59.734223 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:46:59.734644 (+ 421us) client_negotiation.cc:769] Sending connection context
0808 02:46:59.734964 (+ 320us) client_negotiation.cc:240] Negotiation successful
0808 02:46:59.735549 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1248,"thread_start_us":2150,"threads_started":1}
I20260808 02:46:59.739537 14080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:46:59.706083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:36855 (local address 127.25.137.254:34715)
0808 02:46:59.707816 (+ 1733us) server_negotiation.cc:206] Beginning negotiation
0808 02:46:59.707835 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:46:59.710505 (+ 2670us) server_negotiation.cc:407] Connection header received
0808 02:46:59.714444 (+ 3939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:46:59.714465 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:46:59.715063 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:46:59.715553 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:46:59.718145 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:59.719931 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:59.722407 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:46:59.723358 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:46:59.723529 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:46:59.727551 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:46:59.727559 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:46:59.727574 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:46:59.727611 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:46:59.731303 (+ 3692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:59.732363 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:59.732372 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:59.732380 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:59.732624 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:46:59.733220 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:46:59.733225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:46:59.733229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:46:59.733931 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:46:59.734124 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 02:46:59.737863 (+ 3739us) server_negotiation.cc:299] Negotiation successful
0808 02:46:59.738296 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1439,"thread_start_us":1490,"threads_started":1}
I20260808 02:46:59.742758 14077 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34715
I20260808 02:46:59.743103 14077 heartbeater.cc:464] Registering TS with master...
I20260808 02:46:59.744041 14077 heartbeater.cc:511] Master 127.25.137.254:34715 requested a full tablet report, sending...
W20260808 02:46:59.747376 14077 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:34715 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 02:46:59.747634 14077 heartbeater.cc:464] Registering TS with master...
I20260808 02:46:59.747954 14077 heartbeater.cc:511] Master 127.25.137.254:34715 requested a full tablet report, sending...
I20260808 02:46:59.749141 14077 heartbeater.cc:464] Registering TS with master...
I20260808 02:46:59.749395 14077 heartbeater.cc:511] Master 127.25.137.254:34715 requested a full tablet report, sending...
W20260808 02:46:59.750530 14077 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:47:00.167784 14084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:00.168236 14084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:00.168435 14084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:00.180263 14084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:00.180440 14084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:00.180663 14084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:47:00.196689 14084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.137.212:49567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:46211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=44343
--tserver_master_addrs=127.25.137.254:34715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:00.198485 14084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:00.200300 14084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:00.236774 14110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:00.237445 14109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:00.238864 14115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:00.239815 14084 server_base.cc:1034] running on GCE node
I20260808 02:47:00.240667 14084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:00.259958 14084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:00.261343 14084 hybrid_clock.cc:648] HybridClock initialized: now 1786157220261254 us; error 75 us; skew 500 ppm
I20260808 02:47:00.261881 14084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:00 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157220, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:00.266499 14084 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:47:00.270874 14084 webserver.cc:460] Webserver started at http://127.25.137.194:44343/ using document root <none> and password file <none>
I20260808 02:47:00.271838 14084 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:00.271994 14084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:47:00 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157220, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:47:00.300880 14084 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.004s sys 0.004s
I20260808 02:47:00.305629 14131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:00.307463 14084 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.004s
I20260808 02:47:00.307909 14084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "698aa7593bc4449ca84c2cf05152710e"
format_stamp: "Formatted at 2026-08-08 02:46:57 on dist-test-slave-k5rr"
server_key: "4a3493c99595a451dcd4cb2910014f46"
server_key_iv: "dbc0417e056bd929cdda98cc8f8a02ef"
server_key_version: "kuduclusterkey@0"
I20260808 02:47:00.309597 14084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 02:47:00 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786157220, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:00.354713 14084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:00.360177 14084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:00.360556 14084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:00.361873 14084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:00.363559 14084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:00.363662 14084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:00.363780 14084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:00.363867 14084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:00.418308 14084 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:46211
I20260808 02:47:00.418354 14294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:46211 every 8 connection(s)
I20260808 02:47:00.420349 14084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:47:00.428138 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 14084
I20260808 02:47:00.428884 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.137.212:49567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:39741
--local_ip_for_outbound_sockets=127.25.137.195
--tserver_master_addrs=127.25.137.254:34715
--webserver_port=38377
--webserver_interface=127.25.137.195
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.137.212:49567/kms with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:47:00 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157220, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:00.458866 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:00.424355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:57513 (local address 127.25.137.254:34715)
0808 02:47:00.439695 (+ 15340us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:00.439701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:00.439721 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:47:00.439783 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:00.439788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:00.439869 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:00.439983 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:00.442656 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:00.443928 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:00.446379 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:00.447103 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:00.447205 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:00.452155 (+ 4950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:00.452162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:00.452168 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:00.452207 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:00.455455 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:00.456185 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:00.456189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:00.456191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:00.456322 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:00.456814 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:00.456817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:00.456820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:00.457140 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:00.457289 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:00.458210 (+ 921us) server_negotiation.cc:299] Negotiation successful
0808 02:47:00.458452 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15201,"thread_start_us":15736,"threads_started":1}
I20260808 02:47:00.461117 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:00.425108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34715 (local address 127.25.137.194:57513)
0808 02:47:00.436256 (+ 11148us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:00.436319 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:00.438939 (+ 2620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:00.440988 (+ 2049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:00.441001 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:00.441668 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:00.442480 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:00.442502 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:00.444135 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:00.444141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:00.446169 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:00.446195 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:00.447869 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:00.447876 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:00.448167 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:00.449100 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:00.449129 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:00.451897 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:00.455591 (+ 3694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:00.455601 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:00.455618 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:00.456061 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:00.456430 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:00.456434 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:00.456437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:00.456695 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:00.457264 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:00.457272 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:00.457720 (+ 448us) client_negotiation.cc:769] Sending connection context
0808 02:47:00.459179 (+ 1459us) client_negotiation.cc:240] Negotiation successful
0808 02:47:00.459871 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10774,"thread_start_us":14157,"threads_started":1}
I20260808 02:47:00.462628 14295 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34715
I20260808 02:47:00.462971 14295 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:00.463675 14295 heartbeater.cc:511] Master 127.25.137.254:34715 requested a full tablet report, sending...
I20260808 02:47:00.465524 13776 ts_manager.cc:194] Registered new tserver with Master: 698aa7593bc4449ca84c2cf05152710e (127.25.137.194:46211)
I20260808 02:47:00.467145 13776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:57513
I20260808 02:47:00.750703 14077 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:00.751201 14077 heartbeater.cc:511] Master 127.25.137.254:34715 requested a full tablet report, sending...
I20260808 02:47:00.752558 13776 ts_manager.cc:194] Registered new tserver with Master: 4b23f65b36b8482da3b3ca89c5af5c42 (127.25.137.193:41297)
I20260808 02:47:00.753763 13776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:36855
Aug 08 02:47:00 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157220, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
W20260808 02:47:00.897878 14305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:00.898240 14305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:00.898464 14305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:00.910111 14305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:00.910245 14305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:00.910421 14305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:47:00.924530 14305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.137.212:49567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:39741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=38377
--tserver_master_addrs=127.25.137.254:34715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:00.926314 14305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:00.927981 14305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:00.950716 14312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:00.959489 14313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:00.961607 14316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:00.962033 14305 server_base.cc:1034] running on GCE node
I20260808 02:47:00.962844 14305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:00.973440 14305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:00.979192 14305 hybrid_clock.cc:648] HybridClock initialized: now 1786157220979097 us; error 77 us; skew 500 ppm
I20260808 02:47:00.979635 14305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:00 dist-test-slave-k5rr krb5kdc[12133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157220, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:00.984098 14305 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:47:01.000921 14305 webserver.cc:460] Webserver started at http://127.25.137.195:38377/ using document root <none> and password file <none>
I20260808 02:47:01.002048 14305 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:01.002231 14305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:47:01 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157220, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:47:01.040558 14305 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:47:01.061091 14328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:01.063350 14305 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.000s sys 0.019s
I20260808 02:47:01.063661 14305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "7815b26804394592956d84b863992c3d"
format_stamp: "Formatted at 2026-08-08 02:46:57 on dist-test-slave-k5rr"
server_key: "9a2efbc8969a24a5fad70110dcbf71cb"
server_key_iv: "257333fe544e41a12c7c4469398d6d7a"
server_key_version: "kuduclusterkey@0"
I20260808 02:47:01.065241 14305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:01.148818 14305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:01.154862 14305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:01.155272 14305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:01.177686 14305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:01.179943 14305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:01.180117 14305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:01.180275 14305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:01.180389 14305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:47:01 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157220, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for HTTP/127.25.137.212@KRBTEST.COM
I20260808 02:47:01.358783 14305 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:39741
I20260808 02:47:01.358847 14507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:39741 every 8 connection(s)
I20260808 02:47:01.360776 14305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:47:01.367236 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 14305
Aug 08 02:47:01 dist-test-slave-k5rr krb5kdc[12133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157220, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:01.433440 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.370151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:40123 (local address 127.25.137.254:34715)
0808 02:47:01.403770 (+ 33619us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:01.403777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:01.408385 (+ 4608us) server_negotiation.cc:407] Connection header received
0808 02:47:01.408492 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:01.408499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:01.408650 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:01.408795 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:01.414916 (+ 6121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.416231 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.418675 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.419420 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.419524 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.424404 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:01.424412 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:01.424417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:01.424489 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:01.428795 (+ 4306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:01.429872 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:01.429879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:01.429882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:01.430123 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:01.430733 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:01.430836 (+ 103us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:01.430840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:01.431238 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:01.431432 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:01.432662 (+ 1230us) server_negotiation.cc:299] Negotiation successful
0808 02:47:01.432925 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":33482,"thread_start_us":33662,"threads_started":1}
I20260808 02:47:01.436213 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.369161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34715 (local address 127.25.137.195:40123)
0808 02:47:01.404685 (+ 35524us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:01.404777 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:01.407878 (+ 3101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:01.413317 (+ 5439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:01.413332 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:01.413887 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:01.414688 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.414718 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.416447 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.416454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.418489 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.418517 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.420037 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.420046 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.420363 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.421119 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:01.421156 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:01.424095 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:01.428969 (+ 4874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:01.428979 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:01.428996 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:01.429687 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:01.430300 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:01.430306 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:01.430309 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:01.430594 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:01.431569 (+ 975us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:01.431577 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:01.432175 (+ 598us) client_negotiation.cc:769] Sending connection context
0808 02:47:01.434068 (+ 1893us) client_negotiation.cc:240] Negotiation successful
0808 02:47:01.434699 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35090,"thread_start_us":39734,"threads_started":1}
I20260808 02:47:01.438352 14508 heartbeater.cc:347] Connected to a master server at 127.25.137.254:34715
I20260808 02:47:01.438709 14508 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:01.439145 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.369890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46784 (local address 127.25.137.254:34715)
0808 02:47:01.403332 (+ 33442us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:01.403340 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:01.410496 (+ 7156us) server_negotiation.cc:407] Connection header received
0808 02:47:01.410742 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:01.410748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:01.410856 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:01.410982 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:01.413381 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.414603 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.417792 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.418477 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.418618 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.423438 (+ 4820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:01.423445 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:01.423451 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:01.423513 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:01.428785 (+ 5272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:01.431599 (+ 2814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:01.431604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:01.431608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:01.431774 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:01.437148 (+ 5374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:01.437154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:01.437158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:01.437557 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:01.437833 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:01.438502 (+ 669us) server_negotiation.cc:299] Negotiation successful
0808 02:47:01.438742 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":33266,"thread_start_us":33970,"threads_started":1}
I20260808 02:47:01.439766 14508 heartbeater.cc:511] Master 127.25.137.254:34715 requested a full tablet report, sending...
I20260808 02:47:01.443011 13776 ts_manager.cc:194] Registered new tserver with Master: 7815b26804394592956d84b863992c3d (127.25.137.195:39741)
I20260808 02:47:01.444162 13776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:40123
I20260808 02:47:01.447479 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:47:01.470764 14295 heartbeater.cc:503] Master 127.25.137.254:34715 was elected leader, sending a full tablet report...
I20260808 02:47:01.486335 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.466580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46798 (local address 127.25.137.254:34715)
0808 02:47:01.466783 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:01.466790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:01.467364 (+ 574us) server_negotiation.cc:407] Connection header received
0808 02:47:01.467550 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:01.467556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:01.467636 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:01.467722 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:01.469300 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.470024 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.472708 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.473279 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.473357 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.475983 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:01.475988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:01.475991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:01.476024 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:01.483217 (+ 7193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:01.484196 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:01.484201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:01.484205 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:01.484369 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:01.484840 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:01.484846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:01.484850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:01.485191 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:01.485437 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:01.485754 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 02:47:01.485992 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:47:01.492576 13776 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:47:01.504588 13776 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:47:01.558490 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.521229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:39741 (local address 127.0.0.1:56366)
0808 02:47:01.548196 (+ 26967us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:01.548215 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:01.548354 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:01.549372 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:01.549376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:01.549401 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:01.549761 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.549774 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.551971 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.551977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.555239 (+ 3262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.555266 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.557595 (+ 2329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.557601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.557774 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.557810 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:47:01.557888 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 02:47:01.558011 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26817,"mutex_wait_us":757,"thread_start_us":30454,"threads_started":1}
I20260808 02:47:01.559454 14556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.521972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.25.137.195:39741)
0808 02:47:01.544246 (+ 22274us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:01.544264 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:01.548482 (+ 4218us) server_negotiation.cc:407] Connection header received
0808 02:47:01.548587 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:01.548594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:01.548825 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:01.549026 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:01.549993 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.551574 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.555439 (+ 3865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.556941 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.557231 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.557411 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:01.558913 (+ 1502us) server_negotiation.cc:299] Negotiation successful
0808 02:47:01.559101 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22057,"thread_start_us":22304,"threads_started":1}
I20260808 02:47:01.559551 14552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.521943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51058 (local address 127.25.137.194:46211)
0808 02:47:01.546783 (+ 24840us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:01.546791 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:01.546824 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:47:01.546936 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:01.546944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:01.547210 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:01.547445 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:01.548494 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.550708 (+ 2214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.555258 (+ 4550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.556718 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.556905 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.557111 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:01.559026 (+ 1915us) server_negotiation.cc:299] Negotiation successful
0808 02:47:01.559185 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24640,"thread_start_us":24658,"threads_started":1}
I20260808 02:47:01.559980 14549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.520738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:46211 (local address 127.0.0.1:51058)
0808 02:47:01.545209 (+ 24471us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:01.545297 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:01.545534 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:01.547479 (+ 1945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:01.547488 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:01.547526 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:01.548186 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.548201 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.551912 (+ 3711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.551918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.555031 (+ 3113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.555057 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.557286 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.557297 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.557505 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.557570 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 02:47:01.557710 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 02:47:01.557899 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24090,"thread_start_us":31997,"threads_started":1}
I20260808 02:47:01.561507 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.521696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52346 (local address 127.25.137.193:41297)
0808 02:47:01.544247 (+ 22551us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:01.544256 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:01.553179 (+ 8923us) server_negotiation.cc:407] Connection header received
0808 02:47:01.553398 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:01.553407 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:01.553632 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:01.553843 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:01.554813 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.556432 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.558817 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.560099 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.560282 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.560438 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:01.561049 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 02:47:01.561207 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22357,"thread_start_us":22568,"threads_started":1}
I20260808 02:47:01.562227 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.521414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:41297 (local address 127.0.0.1:52346)
0808 02:47:01.553053 (+ 31639us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:01.553072 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:01.553206 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:01.554084 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:01.554092 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:01.554117 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:01.554596 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.554611 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.556614 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.556619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.558629 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.558643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.560650 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.560656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.560837 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.560868 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:47:01.561763 (+ 895us) client_negotiation.cc:240] Negotiation successful
0808 02:47:01.561888 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31482,"mutex_wait_us":1,"thread_start_us":23263,"threads_started":1}
I20260808 02:47:01.564491 14201 tablet_service.cc:1467] Processing CreateTablet for tablet c85425f52a36470f972dd6962bb81566 (DEFAULT_TABLE table=test-table [id=62f2cdcf6073456390b7b0cbcdc15404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:01.564486 14404 tablet_service.cc:1467] Processing CreateTablet for tablet c85425f52a36470f972dd6962bb81566 (DEFAULT_TABLE table=test-table [id=62f2cdcf6073456390b7b0cbcdc15404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:01.565414 13977 tablet_service.cc:1467] Processing CreateTablet for tablet c85425f52a36470f972dd6962bb81566 (DEFAULT_TABLE table=test-table [id=62f2cdcf6073456390b7b0cbcdc15404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:01.566043 14404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85425f52a36470f972dd6962bb81566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:01.566068 14201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85425f52a36470f972dd6962bb81566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:01.566453 13977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85425f52a36470f972dd6962bb81566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:01.587486 14558 tablet_bootstrap.cc:492] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: Bootstrap starting.
I20260808 02:47:01.588277 14557 tablet_bootstrap.cc:492] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: Bootstrap starting.
I20260808 02:47:01.591614 14559 tablet_bootstrap.cc:492] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: Bootstrap starting.
I20260808 02:47:01.592427 14557 tablet_bootstrap.cc:654] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:01.593760 14557 log.cc:824] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:01.595165 14558 tablet_bootstrap.cc:654] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:01.596467 14558 log.cc:824] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:01.597167 14557 tablet_bootstrap.cc:492] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: No bootstrap required, opened a new log
I20260808 02:47:01.597491 14557 ts_tablet_manager.cc:1397] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260808 02:47:01.597852 14559 tablet_bootstrap.cc:654] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:01.599179 14559 log.cc:824] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:01.606029 14558 tablet_bootstrap.cc:492] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: No bootstrap required, opened a new log
I20260808 02:47:01.606401 14558 ts_tablet_manager.cc:1397] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.012s
I20260808 02:47:01.609261 14557 raft_consensus.cc:348] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698aa7593bc4449ca84c2cf05152710e" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 46211 } } peers { permanent_uuid: "7815b26804394592956d84b863992c3d" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 39741 } } peers { permanent_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41297 } }
I20260808 02:47:01.609758 14557 raft_consensus.cc:374] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:01.609884 14557 raft_consensus.cc:729] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7815b26804394592956d84b863992c3d, State: Initialized, Role: FOLLOWER
I20260808 02:47:01.610648 14557 consensus_queue.cc:260] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698aa7593bc4449ca84c2cf05152710e" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 46211 } } peers { permanent_uuid: "7815b26804394592956d84b863992c3d" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 39741 } } peers { permanent_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41297 } }
I20260808 02:47:01.612011 14508 heartbeater.cc:503] Master 127.25.137.254:34715 was elected leader, sending a full tablet report...
I20260808 02:47:01.612581 14557 ts_tablet_manager.cc:1428] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: Time spent starting tablet: real 0.015s user 0.011s sys 0.002s
I20260808 02:47:01.613854 14559 tablet_bootstrap.cc:492] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: No bootstrap required, opened a new log
I20260808 02:47:01.614140 14559 ts_tablet_manager.cc:1397] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.010s
I20260808 02:47:01.618283 14558 raft_consensus.cc:348] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698aa7593bc4449ca84c2cf05152710e" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 46211 } } peers { permanent_uuid: "7815b26804394592956d84b863992c3d" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 39741 } } peers { permanent_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41297 } }
I20260808 02:47:01.618865 14558 raft_consensus.cc:374] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:01.619014 14558 raft_consensus.cc:729] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 698aa7593bc4449ca84c2cf05152710e, State: Initialized, Role: FOLLOWER
I20260808 02:47:01.619726 14558 consensus_queue.cc:260] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698aa7593bc4449ca84c2cf05152710e" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 46211 } } peers { permanent_uuid: "7815b26804394592956d84b863992c3d" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 39741 } } peers { permanent_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41297 } }
I20260808 02:47:01.626555 14558 ts_tablet_manager.cc:1428] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: Time spent starting tablet: real 0.020s user 0.012s sys 0.006s
I20260808 02:47:01.627393 14559 raft_consensus.cc:348] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698aa7593bc4449ca84c2cf05152710e" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 46211 } } peers { permanent_uuid: "7815b26804394592956d84b863992c3d" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 39741 } } peers { permanent_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41297 } }
I20260808 02:47:01.627832 14559 raft_consensus.cc:374] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:01.627944 14559 raft_consensus.cc:729] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b23f65b36b8482da3b3ca89c5af5c42, State: Initialized, Role: FOLLOWER
I20260808 02:47:01.628566 14559 consensus_queue.cc:260] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698aa7593bc4449ca84c2cf05152710e" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 46211 } } peers { permanent_uuid: "7815b26804394592956d84b863992c3d" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 39741 } } peers { permanent_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41297 } }
I20260808 02:47:01.629653 14077 heartbeater.cc:503] Master 127.25.137.254:34715 was elected leader, sending a full tablet report...
I20260808 02:47:01.630362 14559 ts_tablet_manager.cc:1428] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: Time spent starting tablet: real 0.016s user 0.005s sys 0.009s
W20260808 02:47:01.666106 14511 tablet.cc:2369] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:47:01.695029 14297 tablet.cc:2369] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:47:01.716358 14079 tablet.cc:2369] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:47:01.808352 14563 raft_consensus.cc:482] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:47:01.808676 14563 raft_consensus.cc:504] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698aa7593bc4449ca84c2cf05152710e" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 46211 } } peers { permanent_uuid: "7815b26804394592956d84b863992c3d" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 39741 } } peers { permanent_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41297 } }
I20260808 02:47:01.811338 14563 leader_election.cc:288] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 698aa7593bc4449ca84c2cf05152710e (127.25.137.194:46211), 4b23f65b36b8482da3b3ca89c5af5c42 (127.25.137.193:41297)
I20260808 02:47:01.822309 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.812541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:41297 (local address 127.25.137.195:53457)
0808 02:47:01.812713 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:01.812733 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:01.812898 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:01.813317 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:01.813323 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:01.813351 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:01.813680 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.813697 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.815867 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.815873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.819026 (+ 3153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.819042 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.821275 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.821280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.821655 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.821691 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:47:01.821805 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 02:47:01.821937 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":16}
I20260808 02:47:01.822345 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.812142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:53457 (local address 127.25.137.193:41297)
0808 02:47:01.812393 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:01.812397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:01.812983 (+ 586us) server_negotiation.cc:407] Connection header received
0808 02:47:01.813132 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:01.813136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:01.813196 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:01.813274 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:01.813904 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.815612 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.819266 (+ 3654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.821120 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.821306 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.821387 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:01.821853 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 02:47:01.821985 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:47:01.824443 14007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85425f52a36470f972dd6962bb81566" candidate_uuid: "7815b26804394592956d84b863992c3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" is_pre_election: true
I20260808 02:47:01.825067 14552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.812002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:50973 (local address 127.25.137.194:46211)
0808 02:47:01.812309 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:01.812316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:01.814793 (+ 2477us) server_negotiation.cc:407] Connection header received
0808 02:47:01.815056 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:01.815062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:01.815176 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:01.815269 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:01.817044 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.818305 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.822093 (+ 3788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.823788 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.824009 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.824090 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:01.824592 (+ 502us) server_negotiation.cc:299] Negotiation successful
0808 02:47:01.824736 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 02:47:01.825536 14007 raft_consensus.cc:2454] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7815b26804394592956d84b863992c3d in term 0.
I20260808 02:47:01.825861 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.813033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:46211 (local address 127.25.137.195:50973)
0808 02:47:01.814674 (+ 1641us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:01.814692 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:01.814907 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:01.815290 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:01.815295 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:01.815329 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:01.816826 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.816840 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.818516 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.818521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.821848 (+ 3327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:01.821864 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.824114 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.824121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:01.824413 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.824447 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:47:01.824538 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 02:47:01.824660 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1480,"mutex_wait_us":21,"thread_start_us":1206,"threads_started":1}
I20260808 02:47:01.826705 14337 leader_election.cc:302] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b23f65b36b8482da3b3ca89c5af5c42, 7815b26804394592956d84b863992c3d; no voters:
I20260808 02:47:01.827287 14563 raft_consensus.cc:2781] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:47:01.827617 14563 raft_consensus.cc:482] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:47:01.827857 14563 raft_consensus.cc:3037] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:01.827582 14229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85425f52a36470f972dd6962bb81566" candidate_uuid: "7815b26804394592956d84b863992c3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "698aa7593bc4449ca84c2cf05152710e" is_pre_election: true
I20260808 02:47:01.828677 14229 raft_consensus.cc:2454] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7815b26804394592956d84b863992c3d in term 0.
I20260808 02:47:01.831725 14563 raft_consensus.cc:504] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698aa7593bc4449ca84c2cf05152710e" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 46211 } } peers { permanent_uuid: "7815b26804394592956d84b863992c3d" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 39741 } } peers { permanent_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41297 } }
I20260808 02:47:01.832624 14563 leader_election.cc:288] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [CANDIDATE]: Term 1 election: Requested vote from peers 698aa7593bc4449ca84c2cf05152710e (127.25.137.194:46211), 4b23f65b36b8482da3b3ca89c5af5c42 (127.25.137.193:41297)
I20260808 02:47:01.833201 14007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85425f52a36470f972dd6962bb81566" candidate_uuid: "7815b26804394592956d84b863992c3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b23f65b36b8482da3b3ca89c5af5c42"
I20260808 02:47:01.833215 14229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85425f52a36470f972dd6962bb81566" candidate_uuid: "7815b26804394592956d84b863992c3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "698aa7593bc4449ca84c2cf05152710e"
I20260808 02:47:01.833441 14007 raft_consensus.cc:3037] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:01.833544 14229 raft_consensus.cc:3037] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:01.837143 14007 raft_consensus.cc:2454] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7815b26804394592956d84b863992c3d in term 1.
I20260808 02:47:01.837603 14337 leader_election.cc:302] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b23f65b36b8482da3b3ca89c5af5c42, 7815b26804394592956d84b863992c3d; no voters:
I20260808 02:47:01.837911 14563 raft_consensus.cc:2781] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:47:01.838593 14229 raft_consensus.cc:2454] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7815b26804394592956d84b863992c3d in term 1.
I20260808 02:47:01.839202 14563 raft_consensus.cc:686] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 1 LEADER]: Becoming Leader. State: Replica: 7815b26804394592956d84b863992c3d, State: Running, Role: LEADER
I20260808 02:47:01.839953 14563 consensus_queue.cc:237] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698aa7593bc4449ca84c2cf05152710e" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 46211 } } peers { permanent_uuid: "7815b26804394592956d84b863992c3d" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 39741 } } peers { permanent_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41297 } }
I20260808 02:47:01.846859 13776 catalog_manager.cc:5495] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7815b26804394592956d84b863992c3d (127.25.137.195). New cstate: current_term: 1 leader_uuid: "7815b26804394592956d84b863992c3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698aa7593bc4449ca84c2cf05152710e" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 46211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7815b26804394592956d84b863992c3d" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 39741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260808 02:47:01.936223 14556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.928516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41894 (local address 127.25.137.195:39741)
0808 02:47:01.928763 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:01.928770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:01.929034 (+ 264us) server_negotiation.cc:407] Connection header received
0808 02:47:01.929279 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:01.929283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:01.929365 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:01.929498 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:47:01.930048 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.931233 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.933376 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:01.934112 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:01.934227 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:01.934977 (+ 750us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:47:01.935543 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:47:01.935679 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:01.935797 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 02:47:01.935882 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:47:01.945833 14229 raft_consensus.cc:1261] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e [term 1 FOLLOWER]: Refusing update from remote peer 7815b26804394592956d84b863992c3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:47:01.945953 14007 raft_consensus.cc:1261] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42 [term 1 FOLLOWER]: Refusing update from remote peer 7815b26804394592956d84b863992c3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:47:01.946810 14575 consensus_queue.cc:1035] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "698aa7593bc4449ca84c2cf05152710e" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 46211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:47:01.947448 14563 consensus_queue.cc:1035] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b23f65b36b8482da3b3ca89c5af5c42" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:47:01.961375 14580 mvcc.cc:205] Tried to move back new op lower bound from 7316099981078433792 to 7316099980661440512. Current Snapshot: MvccSnapshot[applied={T|T < 7316099981078433792}]
I20260808 02:47:01.982385 13776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:47:01.982739 13776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:47:01.988373 13776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df81646c8d6644eb9431cc626b4eae72: Sending DeleteTablet for 3 replicas of tablet c85425f52a36470f972dd6962bb81566
I20260808 02:47:01.989735 14201 tablet_service.cc:1514] Processing DeleteTablet for tablet c85425f52a36470f972dd6962bb81566 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:01 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:51058
I20260808 02:47:01.990114 13977 tablet_service.cc:1514] Processing DeleteTablet for tablet c85425f52a36470f972dd6962bb81566 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:01 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:52346
I20260808 02:47:01.991284 14600 tablet_replica.cc:330] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: stopping tablet replica
I20260808 02:47:01.991382 14404 tablet_service.cc:1514] Processing DeleteTablet for tablet c85425f52a36470f972dd6962bb81566 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:01 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:56366
I20260808 02:47:01.991557 14601 tablet_replica.cc:330] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: stopping tablet replica
I20260808 02:47:01.991914 14600 raft_consensus.cc:2229] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:47:01.991945 14601 raft_consensus.cc:2229] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:47:01.992456 14601 raft_consensus.cc:2258] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:01.992502 14600 raft_consensus.cc:2258] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:01.993412 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 13836
I20260808 02:47:01.994724 14601 ts_tablet_manager.cc:1905] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:01.994781 14600 ts_tablet_manager.cc:1905] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:02.002076 14600 ts_tablet_manager.cc:1918] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:02.002159 14601 ts_tablet_manager.cc:1918] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:02.002326 14601 log.cc:1196] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/c85425f52a36470f972dd6962bb81566
I20260808 02:47:02.002480 14600 log.cc:1196] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/c85425f52a36470f972dd6962bb81566
I20260808 02:47:02.002849 14601 ts_tablet_manager.cc:1939] T c85425f52a36470f972dd6962bb81566 P 4b23f65b36b8482da3b3ca89c5af5c42: Deleting consensus metadata
I20260808 02:47:02.003165 14600 ts_tablet_manager.cc:1939] T c85425f52a36470f972dd6962bb81566 P 698aa7593bc4449ca84c2cf05152710e: Deleting consensus metadata
I20260808 02:47:02.004707 13758 catalog_manager.cc:4841] TS 4b23f65b36b8482da3b3ca89c5af5c42 (127.25.137.193:41297): tablet c85425f52a36470f972dd6962bb81566 (table test-table [id=62f2cdcf6073456390b7b0cbcdc15404]) successfully deleted
I20260808 02:47:02.005093 13761 catalog_manager.cc:4841] TS 698aa7593bc4449ca84c2cf05152710e (127.25.137.194:46211): tablet c85425f52a36470f972dd6962bb81566 (table test-table [id=62f2cdcf6073456390b7b0cbcdc15404]) successfully deleted
I20260808 02:47:02.010421 14603 tablet_replica.cc:330] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: stopping tablet replica
I20260808 02:47:02.010818 14603 raft_consensus.cc:2229] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:47:02.011499 14603 raft_consensus.cc:2258] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:02.014142 14603 ts_tablet_manager.cc:1905] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:02.022022 14603 ts_tablet_manager.cc:1918] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:02.022248 14603 log.cc:1196] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/c85425f52a36470f972dd6962bb81566
I20260808 02:47:02.022914 14603 ts_tablet_manager.cc:1939] T c85425f52a36470f972dd6962bb81566 P 7815b26804394592956d84b863992c3d: Deleting consensus metadata
I20260808 02:47:02.024989 13753 catalog_manager.cc:4841] TS 7815b26804394592956d84b863992c3d (127.25.137.195:39741): tablet c85425f52a36470f972dd6962bb81566 (table test-table [id=62f2cdcf6073456390b7b0cbcdc15404]) successfully deleted
I20260808 02:47:02.028446 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:01.996236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60592 (local address 127.25.137.193:41297)
0808 02:47:01.996407 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:01.996413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:02.007844 (+ 11431us) server_negotiation.cc:407] Connection header received
0808 02:47:02.008102 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:02.008110 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:02.008213 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:02.008341 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:02.010172 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:02.011159 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:02.014581 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:02.015626 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:02.015892 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:02.019976 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:02.019997 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:02.020016 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:02.020072 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:02.025050 (+ 4978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:02.026170 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:02.026178 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:02.026180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:02.026337 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:02.026748 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:02.026751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:02.026753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:02.027139 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:02.027401 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:02.027790 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 02:47:02.028052 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:47:02.029114 14072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:02.154707 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 13836
I20260808 02:47:02.180841 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 14084
I20260808 02:47:02.195937 14552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:02.181666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38356 (local address 127.25.137.194:46211)
0808 02:47:02.181868 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:02.181874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:02.181896 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:47:02.182055 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:02.182060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:02.182166 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:02.182286 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:02.183611 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:02.184680 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:02.186069 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:02.186685 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:02.186828 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:02.189367 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:02.189387 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:02.189408 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:02.189450 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:02.193119 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:02.193778 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:02.193795 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:02.193799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:02.193972 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:02.194285 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:02.194290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:02.194293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:02.194731 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:02.195034 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:02.195248 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 02:47:02.195481 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:47:02.196624 14287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:02.287104 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 14084
I20260808 02:47:02.308619 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 14305
I20260808 02:47:02.323083 14556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:02.309539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41906 (local address 127.25.137.195:39741)
0808 02:47:02.309723 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:02.309728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:02.309919 (+ 191us) server_negotiation.cc:407] Connection header received
0808 02:47:02.310107 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:02.310110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:02.310186 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:02.310280 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:02.311678 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:02.312501 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:02.313847 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:02.314344 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:02.314472 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:02.317116 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:02.317129 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:02.317145 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:02.317186 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:02.320237 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:02.320999 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:02.321029 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:02.321033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:02.321186 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:02.321572 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:02.321576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:02.321579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:02.321914 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:02.322262 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:02.322517 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 02:47:02.322741 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:47:02.323830 14501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:02.451236 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 14305
I20260808 02:47:02.479719 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 13728
I20260808 02:47:02.480937 13824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:02.594938 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 13728
I20260808 02:47:02.625149 26151 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 02:47:02.970114 26151 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 02:47:02.970345 26151 mini_ranger.cc:66] Stopping Ranger...
I20260808 02:47:03.315587 26151 mini_ranger.cc:68] Stopped Ranger
2026-08-08 02:47:03.315 UTC [12177] LOG: received smart shutdown request
2026-08-08 02:47:03.318 UTC [12177] LOG: background worker "logical replication launcher" (PID 12190) exited with exit code 1
2026-08-08 02:47:03.319 UTC [12183] LOG: shutting down
2026-08-08 02:47:03.462 UTC [12177] LOG: database system is shut down
2026-08-08T02:47:03Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53211 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:47:03 dist-test-slave-k5rr krb5kdc[14627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:47:03 dist-test-slave-k5rr krb5kdc[14627](info): set up 2 sockets
Aug 08 02:47:03 dist-test-slave-k5rr krb5kdc[14627](info): commencing operation
krb5kdc: starting...
W20260808 02:47:04.821352 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:47:04 dist-test-slave-k5rr krb5kdc[14627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:47:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:47:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:47:05.101333 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:44229
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:36947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:44229,127.25.137.253:37529,127.25.137.252:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:05.449088 14651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:05.449436 14651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:05.449579 14651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:05.456905 14651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:47:05.457044 14651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:05.457129 14651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:05.457206 14651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:47:05.457267 14651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:47:05.467329 14651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:36947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:44229,127.25.137.253:37529,127.25.137.252:36487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:44229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:05.468730 14651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:05.470093 14651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:05.482461 14659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:05.482534 14657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:05.484262 14662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:05.487092 14651 server_base.cc:1034] running on GCE node
I20260808 02:47:05.488108 14651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:05.489895 14651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:05.491118 14651 hybrid_clock.cc:648] HybridClock initialized: now 1786157225491001 us; error 115 us; skew 500 ppm
I20260808 02:47:05.491460 14651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:05 dist-test-slave-k5rr krb5kdc[14627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157225, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:05.495077 14651 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:47:05.496990 14651 webserver.cc:460] Webserver started at http://127.25.137.254:36711/ using document root <none> and password file <none>
I20260808 02:47:05.497891 14651 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:05.498075 14651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:05.498461 14651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:05.503183 14651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "51750762300744db8fa07f4a05b99e19"
format_stamp: "Formatted at 2026-08-08 02:47:05 on dist-test-slave-k5rr"
server_key: "2c44aaa66d9cdb7ce9ab3b685c287d89"
server_key_iv: "844b1fa7c2f04b33c9b1a972fabde557"
server_key_version: "encryptionkey@0"
I20260808 02:47:05.504485 14651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "51750762300744db8fa07f4a05b99e19"
format_stamp: "Formatted at 2026-08-08 02:47:05 on dist-test-slave-k5rr"
server_key: "2c44aaa66d9cdb7ce9ab3b685c287d89"
server_key_iv: "844b1fa7c2f04b33c9b1a972fabde557"
server_key_version: "encryptionkey@0"
I20260808 02:47:05.511581 14651 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260808 02:47:05.516070 14671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:05.517681 14651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 02:47:05.517838 14651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "51750762300744db8fa07f4a05b99e19"
format_stamp: "Formatted at 2026-08-08 02:47:05 on dist-test-slave-k5rr"
server_key: "2c44aaa66d9cdb7ce9ab3b685c287d89"
server_key_iv: "844b1fa7c2f04b33c9b1a972fabde557"
server_key_version: "encryptionkey@0"
I20260808 02:47:05.518057 14651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:05.545320 14651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:05.549671 14651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:05.549952 14651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:05.569305 14651 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:44229
I20260808 02:47:05.569311 14764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:44229 every 8 connection(s)
I20260808 02:47:05.570868 14651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:47:05.574712 14765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:05.579380 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 14651
I20260808 02:47:05.579613 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:47:05.580489 26151 external_mini_cluster.cc:1428] Setting key 066e808c47b6f156c3811142760257a3
I20260808 02:47:05.582363 14765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:47:05 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157225, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:05.607368 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.586239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:44229 (local address 127.0.0.1:41548)
0808 02:47:05.587231 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:05.587281 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:05.589762 (+ 2481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:05.590849 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:05.590856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:05.591640 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:05.592115 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:05.592138 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:05.593909 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:05.593913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:05.595034 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:05.595048 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:05.596050 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:05.596058 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:05.596252 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:05.596979 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:05.597030 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:05.599526 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:05.602985 (+ 3459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:05.602998 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:05.603012 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:05.603607 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:05.604105 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:05.604111 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:05.604115 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:05.604327 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:05.605173 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:05.605182 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:05.605464 (+ 282us) client_negotiation.cc:769] Sending connection context
0808 02:47:05.605763 (+ 299us) client_negotiation.cc:240] Negotiation successful
0808 02:47:05.606372 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":82,"thread_start_us":426,"threads_started":1}
I20260808 02:47:05.607399 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.586239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41548 (local address 127.25.137.254:44229)
0808 02:47:05.588168 (+ 1929us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:05.588174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:05.588220 (+ 46us) server_negotiation.cc:407] Connection header received
0808 02:47:05.590308 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:05.590315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:05.590575 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:05.590815 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:05.592345 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:05.593702 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:05.595183 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:05.595796 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:05.595987 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:05.599796 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:05.599805 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:05.599818 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:05.599851 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:05.602835 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:05.603779 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:05.603789 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:05.603792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:05.603972 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:05.604431 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:05.604434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:05.604436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:05.604984 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:05.605186 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:05.605904 (+ 718us) server_negotiation.cc:299] Negotiation successful
0808 02:47:05.606207 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1734,"spinlock_wait_cycles":47488,"thread_start_us":1466,"threads_started":1}
WARNING: no policy specified for kudu/127.25.137.253@KRBTEST.COM; defaulting to no policy
I20260808 02:47:05.610558 14765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.253@KRBTEST.COM" created.
I20260808 02:47:05.611609 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.611179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57054)
0808 02:47:05.611355 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:05.611474 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 02:47:05.611881 14679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
I20260808 02:47:05.613554 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.613176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57060)
0808 02:47:05.613339 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:05.613418 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:47:05.613797 14765 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:37529: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab.
Entry for principal kudu/127.25.137.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab.
I20260808 02:47:05.643631 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.253:37529
--webserver_interface=127.25.137.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab
--principal=kudu/127.25.137.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:36947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:44229,127.25.137.253:37529,127.25.137.252:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:47:05.663264 14765 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } attempt: 1
I20260808 02:47:05.664572 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.664058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57068)
0808 02:47:05.664300 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:05.664436 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 02:47:05.665741 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.665346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57072)
0808 02:47:05.665525 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:05.665601 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 02:47:05.665962 14765 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:37529: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
I20260808 02:47:05.734234 14765 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } attempt: 2
I20260808 02:47:05.735440 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.734905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57080)
0808 02:47:05.735189 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:05.735285 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 02:47:05.736632 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.736246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57096)
0808 02:47:05.736421 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:05.736493 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 02:47:05.736867 14765 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:37529: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
I20260808 02:47:05.828121 14765 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } attempt: 3
I20260808 02:47:05.829222 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.828732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57112)
0808 02:47:05.828927 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:05.829078 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 02:47:05.830296 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.829930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57118)
0808 02:47:05.830082 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:05.830151 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 02:47:05.830469 14765 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:37529: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
I20260808 02:47:05.973718 14765 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } attempt: 4
I20260808 02:47:05.974973 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.974424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57132)
0808 02:47:05.974721 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:05.974817 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260808 02:47:05.976058 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:05.975681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57142)
0808 02:47:05.975846 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:05.975937 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 02:47:05.976254 14765 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:37529: Network error: Client connection negotiation failed: client connection to 127.25.137.253:37529: connect: Connection refused (error 111)
W20260808 02:47:05.977046 14774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:05.977331 14774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:05.977463 14774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:05.984747 14774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:47:05.984865 14774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:05.984959 14774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:05.985054 14774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:47:05.985119 14774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:47:05.995215 14774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:36947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:44229,127.25.137.253:37529,127.25.137.252:36487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab
--principal=kudu/127.25.137.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.253:37529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.137.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:05.996563 14774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:05.997891 14774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:06.009297 14783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:06.009393 14781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:06.009250 14788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:06.010226 14774 server_base.cc:1034] running on GCE node
I20260808 02:47:06.011063 14774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:06.012480 14774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:06.013759 14774 hybrid_clock.cc:648] HybridClock initialized: now 1786157226013710 us; error 38 us; skew 500 ppm
I20260808 02:47:06.014134 14774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157226, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:06.017402 14774 init.cc:377] Logged in from keytab as kudu/127.25.137.253@KRBTEST.COM (short username kudu)
I20260808 02:47:06.018882 14774 webserver.cc:460] Webserver started at http://127.25.137.253:41381/ using document root <none> and password file <none>
I20260808 02:47:06.019554 14774 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:06.019673 14774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:06.019966 14774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:06.023178 14774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/data/instance:
uuid: "e08f948acfe447878e6dd0df8d3f206c"
format_stamp: "Formatted at 2026-08-08 02:47:06 on dist-test-slave-k5rr"
server_key: "1ffac328768e8c24ce0210743a6dec2b"
server_key_iv: "1fe8e4b6fc7bfb7f45a4ad4af0330d4e"
server_key_version: "encryptionkey@0"
I20260808 02:47:06.023991 14774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/wal/instance:
uuid: "e08f948acfe447878e6dd0df8d3f206c"
format_stamp: "Formatted at 2026-08-08 02:47:06 on dist-test-slave-k5rr"
server_key: "1ffac328768e8c24ce0210743a6dec2b"
server_key_iv: "1fe8e4b6fc7bfb7f45a4ad4af0330d4e"
server_key_version: "encryptionkey@0"
I20260808 02:47:06.029942 14774 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260808 02:47:06.033416 14800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:06.034711 14774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 02:47:06.034906 14774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/wal
uuid: "e08f948acfe447878e6dd0df8d3f206c"
format_stamp: "Formatted at 2026-08-08 02:47:06 on dist-test-slave-k5rr"
server_key: "1ffac328768e8c24ce0210743a6dec2b"
server_key_iv: "1fe8e4b6fc7bfb7f45a4ad4af0330d4e"
server_key_version: "encryptionkey@0"
I20260808 02:47:06.035090 14774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:06.049294 14774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:06.053571 14774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:06.053839 14774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:06.072029 14774 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.253:37529
I20260808 02:47:06.072068 14871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.253:37529 every 8 connection(s)
I20260808 02:47:06.073433 14774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
I20260808 02:47:06.077252 14872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:06.082005 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 14774
I20260808 02:47:06.082177 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-1/wal/instance
I20260808 02:47:06.082645 26151 external_mini_cluster.cc:1428] Setting key 35d0e9025ca4a60ee4283a5e1047c601
I20260808 02:47:06.083721 14872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157226, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:06.105214 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.086333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41552 (local address 127.25.137.254:44229)
0808 02:47:06.086513 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.086518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.087954 (+ 1436us) server_negotiation.cc:407] Connection header received
0808 02:47:06.089913 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.089918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.089996 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.090087 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.091651 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.092331 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.093942 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.094552 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.094679 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.098561 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.098567 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.098570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.098611 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.101607 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.102585 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.102590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.102592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.102733 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.103252 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.103256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.103258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.103590 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.103725 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.104681 (+ 956us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.104918 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:47:06.106546 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.086946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:44229 (local address 127.0.0.1:41552)
0808 02:47:06.087774 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.087813 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:06.089612 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:06.090331 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:06.090340 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:06.090872 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:06.091451 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.091471 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.092493 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.092498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.093777 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.093791 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.094796 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.094802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.095041 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.095678 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:06.095701 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:06.098261 (+ 2560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:06.101761 (+ 3500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.101770 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.101788 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.102434 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.102897 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.102902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.102905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.103134 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.103729 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:06.103737 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:06.104193 (+ 456us) client_negotiation.cc:769] Sending connection context
0808 02:47:06.104539 (+ 346us) client_negotiation.cc:240] Negotiation successful
0808 02:47:06.105083 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":369,"threads_started":1}
I20260808 02:47:06.108393 14872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.137.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.252@KRBTEST.COM" created.
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157226, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
I20260808 02:47:06.123786 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.109072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57146)
0808 02:47:06.109234 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.109257 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:06.109353 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:06.110484 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:06.110488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:06.110981 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:06.111115 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.111124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.112519 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.112524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.113639 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.113654 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.114456 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.114466 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.114663 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.115275 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:06.115299 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:06.117599 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:06.121213 (+ 3614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.121218 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.121221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.121657 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.122053 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.122057 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.122061 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.122223 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.122993 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:06.122997 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:06.123125 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 02:47:06.123237 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 02:47:06.123470 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 02:47:06.124397 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.109416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57146 (local address 127.25.137.253:37529)
0808 02:47:06.110065 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.110073 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.110101 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:47:06.110147 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.110152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.110338 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.110472 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.111269 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.112389 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.113780 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.114307 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.114408 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.117840 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.117855 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.117872 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.117907 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.121087 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.121807 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.121816 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.121819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.121957 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.122385 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.122390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.122393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.122875 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.123055 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.123423 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.123671 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":365,"threads_started":1}
I20260808 02:47:06.126783 14872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:47:06.128021 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.127541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39456)
0808 02:47:06.127727 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.127868 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260808 02:47:06.128324 14801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
I20260808 02:47:06.129947 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.129570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39466)
0808 02:47:06.129757 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.129837 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 02:47:06.130162 14872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:36487: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab.
Entry for principal kudu/127.25.137.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab.
I20260808 02:47:06.146576 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.252:36487
--webserver_interface=127.25.137.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab
--principal=kudu/127.25.137.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:36947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:44229,127.25.137.253:37529,127.25.137.252:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:47:06.179507 14872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } attempt: 1
I20260808 02:47:06.180739 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.180278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39468)
0808 02:47:06.180472 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.180585 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 02:47:06.181838 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.181482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39476)
0808 02:47:06.181627 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.181697 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 02:47:06.182056 14872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:36487: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
I20260808 02:47:06.250280 14872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } attempt: 2
I20260808 02:47:06.251475 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.250990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39486)
0808 02:47:06.251191 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.251314 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 02:47:06.252512 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.252163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39492)
0808 02:47:06.252320 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.252399 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 02:47:06.252717 14872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:36487: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
I20260808 02:47:06.275532 14765 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } attempt: 5
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157225, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
I20260808 02:47:06.290068 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.276398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57148 (local address 127.25.137.253:37529)
0808 02:47:06.276555 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.276559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.276579 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:47:06.276840 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.276845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.276949 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.277105 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.278140 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.278851 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.280505 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.280975 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.281114 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.284695 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.284700 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.284704 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.284737 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.287326 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.288144 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.288149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.288152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.288286 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.288675 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.288678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.288680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.288969 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.289120 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.289599 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.289845 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:47:06.290162 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.276269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57148)
0808 02:47:06.276490 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.276513 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:06.276651 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:06.277159 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:06.277165 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:06.277680 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:06.277953 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.277968 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.278992 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.278998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.280338 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.280356 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.281148 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.281153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.281338 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.281976 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:06.281999 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:06.284499 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:06.287469 (+ 2970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.287474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.287478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.288004 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.288411 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.288416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.288420 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.288585 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.289119 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:06.289124 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:06.289305 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 02:47:06.289480 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 02:47:06.289816 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 02:47:06.291122 14765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:47:06.292315 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.291914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39506)
0808 02:47:06.292084 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.292162 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 02:47:06.293319 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.292951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39522)
0808 02:47:06.293120 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.293187 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260808 02:47:06.293525 14765 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:36487: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
I20260808 02:47:06.343918 14872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } attempt: 3
I20260808 02:47:06.344727 14765 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } attempt: 1
I20260808 02:47:06.345244 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.344645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39536)
0808 02:47:06.344924 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.345074 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 02:47:06.345858 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.345366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39540)
0808 02:47:06.345578 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.345700 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260808 02:47:06.346566 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.346024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39552)
0808 02:47:06.346321 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.346418 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200}
W20260808 02:47:06.346755 14872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:36487: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
I20260808 02:47:06.347048 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.346528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39554)
0808 02:47:06.346787 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.346901 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 02:47:06.347307 14765 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:36487: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
I20260808 02:47:06.415547 14765 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } attempt: 2
I20260808 02:47:06.416888 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.416354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39562)
0808 02:47:06.416584 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.416684 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 02:47:06.417909 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.417573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39574)
0808 02:47:06.417689 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.417793 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 02:47:06.418129 14765 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:36487: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
W20260808 02:47:06.486146 14881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:06.486434 14881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:06.486565 14881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 02:47:06.489969 14872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } attempt: 4
I20260808 02:47:06.491250 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.490698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39580)
0808 02:47:06.490968 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.491070 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260808 02:47:06.492391 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.492020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39592)
0808 02:47:06.492185 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.492273 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 02:47:06.492614 14872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:36487: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
W20260808 02:47:06.494369 14881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:47:06.494477 14881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:06.494549 14881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:06.494624 14881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:47:06.494683 14881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:47:06.504558 14881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:36947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:44229,127.25.137.253:37529,127.25.137.252:36487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab
--principal=kudu/127.25.137.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.252:36487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.137.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:06.505923 14881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:06.507213 14881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:06.518771 14890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:06.518865 14894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:06.518888 14891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:06.519809 14881 server_base.cc:1034] running on GCE node
I20260808 02:47:06.520531 14881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:06.522037 14881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:06.523252 14881 hybrid_clock.cc:648] HybridClock initialized: now 1786157226523191 us; error 51 us; skew 500 ppm
I20260808 02:47:06.523699 14881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:47:06.524330 14765 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } attempt: 3
I20260808 02:47:06.525502 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.524961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39604)
0808 02:47:06.525191 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.525292 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 02:47:06.526600 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.526172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39614)
0808 02:47:06.526369 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.526463 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260808 02:47:06.526837 14765 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:36487: Network error: Client connection negotiation failed: client connection to 127.25.137.252:36487: connect: Connection refused (error 111)
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157226, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:06.527491 14881 init.cc:377] Logged in from keytab as kudu/127.25.137.252@KRBTEST.COM (short username kudu)
I20260808 02:47:06.528903 14881 webserver.cc:460] Webserver started at http://127.25.137.252:38371/ using document root <none> and password file <none>
I20260808 02:47:06.529610 14881 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:06.529713 14881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:06.529964 14881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:06.532944 14881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/data/instance:
uuid: "03f7fa69e79c4247a8e4f3aa4a471f01"
format_stamp: "Formatted at 2026-08-08 02:47:06 on dist-test-slave-k5rr"
server_key: "5ea5d554a6884986f98094206b665dd1"
server_key_iv: "050c74bd9091a8223aa07623a75afc49"
server_key_version: "encryptionkey@0"
I20260808 02:47:06.533742 14881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/wal/instance:
uuid: "03f7fa69e79c4247a8e4f3aa4a471f01"
format_stamp: "Formatted at 2026-08-08 02:47:06 on dist-test-slave-k5rr"
server_key: "5ea5d554a6884986f98094206b665dd1"
server_key_iv: "050c74bd9091a8223aa07623a75afc49"
server_key_version: "encryptionkey@0"
I20260808 02:47:06.538841 14881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:47:06.542248 14903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:06.543594 14881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:47:06.543748 14881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/wal
uuid: "03f7fa69e79c4247a8e4f3aa4a471f01"
format_stamp: "Formatted at 2026-08-08 02:47:06 on dist-test-slave-k5rr"
server_key: "5ea5d554a6884986f98094206b665dd1"
server_key_iv: "050c74bd9091a8223aa07623a75afc49"
server_key_version: "encryptionkey@0"
I20260808 02:47:06.543926 14881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:06.556321 14881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:06.560083 14881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:06.560345 14881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:06.578672 14881 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.252:36487
I20260808 02:47:06.578725 14983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.252:36487 every 8 connection(s)
I20260808 02:47:06.579994 14881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
I20260808 02:47:06.583184 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 14881
I20260808 02:47:06.583364 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157019169307-26151-0/minicluster-data/master-2/wal/instance
I20260808 02:47:06.583755 14984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:06.583907 26151 external_mini_cluster.cc:1428] Setting key 748fff7e8ca263acd3aabe0a414c77fb
I20260808 02:47:06.593588 14984 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157226, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:06.608098 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.585701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39622 (local address 127.25.137.252:36487)
0808 02:47:06.586571 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.586585 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.586641 (+ 56us) server_negotiation.cc:407] Connection header received
0808 02:47:06.588606 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.588633 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.589148 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.589574 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.590900 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.592849 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.594404 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.595360 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.595535 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.600832 (+ 5297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.600840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.600850 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.600892 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.604037 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.604916 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.604925 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.604933 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.605190 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.605608 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.605611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.605613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.606096 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.606318 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.606821 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.607180 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"spinlock_wait_cycles":17280,"thread_start_us":361,"threads_started":1}
I20260808 02:47:06.611142 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.596173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:44229 (local address 127.0.0.1:41556)
0808 02:47:06.596941 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.597061 (+ 120us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:06.597238 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:06.597660 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:06.597668 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:06.598511 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:06.598899 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.598915 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.600181 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.600187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.601300 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.601313 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.602030 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.602037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.602207 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.602919 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:06.602937 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:06.604999 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:06.608552 (+ 3553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.608561 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.608565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.608967 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.609455 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.609460 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.609463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.609622 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.610120 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:06.610127 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:06.610336 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 02:47:06.610503 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 02:47:06.610774 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":321,"threads_started":1}
I20260808 02:47:06.611241 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.596312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41556 (local address 127.25.137.254:44229)
0808 02:47:06.596528 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.596534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.597155 (+ 621us) server_negotiation.cc:407] Connection header received
0808 02:47:06.597400 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.597404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.597468 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.597548 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.599064 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.600031 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.601437 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.601846 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.601961 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.605213 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.605219 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.605222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.605275 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.608430 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.609115 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.609120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.609124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.609317 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.609729 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.609732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.609735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.609995 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.610121 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.610652 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.610871 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 02:47:06.612789 14984 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157226, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:06.626178 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.613444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:37529 (local address 127.0.0.1:57156)
0808 02:47:06.613676 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.613693 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:06.613778 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:06.614284 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:06.614287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:06.614605 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:06.614727 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.614735 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.615677 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.615681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.616554 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.616565 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.617304 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.617309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.617477 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.617942 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:06.617958 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:06.619977 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:06.623631 (+ 3654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.623637 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.623640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.624146 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.624522 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.624527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.624531 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.624697 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.625352 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:06.625355 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:06.625481 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 02:47:06.625644 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 02:47:06.625916 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260808 02:47:06.626266 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.613713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57156 (local address 127.25.137.253:37529)
0808 02:47:06.613965 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.613972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.613993 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:47:06.614059 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.614064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.614145 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.614270 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.614891 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.615548 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.616686 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.617147 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.617253 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.620178 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.620186 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.620191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.620245 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.623492 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.624263 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.624269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.624272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.624416 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.624845 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.624850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.624853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.625242 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.625394 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.625736 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.625958 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260808 02:47:06.627089 14984 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:47:06.627576 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.611409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41568 (local address 127.25.137.254:44229)
0808 02:47:06.611711 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.611716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.611733 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:47:06.611948 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.611953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.612028 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.612131 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.613689 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.614699 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.616149 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.616601 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.616723 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.621546 (+ 4823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.621551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.621554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.621583 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.624721 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.625554 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.625559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.625563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.625718 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.626115 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.626119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.626122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.626469 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.626736 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.627049 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.627253 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":400,"threads_started":1}
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157226, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
I20260808 02:47:06.640957 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.628248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39630 (local address 127.25.137.252:36487)
0808 02:47:06.628400 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.628406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.628422 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:47:06.628462 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.628465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.628512 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.628580 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.629405 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.630210 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.631635 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.632206 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.632300 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.635437 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.635441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.635445 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.635473 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.638267 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.639007 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.639011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.639013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.639151 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.639551 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.639555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.639557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.639816 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.639946 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.640379 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.640690 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":52}
I20260808 02:47:06.640980 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.627829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39630)
0808 02:47:06.627997 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.628015 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:06.628118 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:06.628628 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:06.628637 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:06.629040 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:06.629268 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.629280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.630352 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.630358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.631500 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.631513 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.632384 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.632390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.632564 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.633118 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:06.633141 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:06.635270 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:06.638398 (+ 3128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.638403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.638406 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.638884 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.639253 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.639258 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.639261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.639444 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.639936 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:06.639941 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:06.640083 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 02:47:06.640257 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 02:47:06.640552 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 02:47:06.644424 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.629151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57164 (local address 127.25.137.253:37529)
0808 02:47:06.629299 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.629304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.629343 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:47:06.629520 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.629525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.629593 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.629688 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.631069 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.631901 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.633537 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.633933 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.634022 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.638458 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.638464 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.638469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.638504 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.641842 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.642626 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.642630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.642632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.642761 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.643119 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.643123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.643125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.643379 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.643624 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.643921 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.644133 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:47:06.646205 14984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01: Bootstrap starting.
I20260808 02:47:06.649446 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:47:06.649992 14984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:06.650991 14984 log.cc:824] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:06.653801 14984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01: No bootstrap required, opened a new log
I20260808 02:47:06.664878 14984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } }
I20260808 02:47:06.665359 14984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:06.665473 14984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03f7fa69e79c4247a8e4f3aa4a471f01, State: Initialized, Role: FOLLOWER
I20260808 02:47:06.667104 14984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } }
I20260808 02:47:06.673419 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.654160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41570 (local address 127.25.137.254:44229)
0808 02:47:06.654332 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.654338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.655311 (+ 973us) server_negotiation.cc:407] Connection header received
0808 02:47:06.655574 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.655580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.655662 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.655735 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.658248 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.659418 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.661707 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.662425 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.662524 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.666383 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.666388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.666391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.666424 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.669914 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.670902 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.670908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.670911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.671070 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.671576 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.671581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.671584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.671924 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.672159 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.672758 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.672995 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:47:06.674059 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.654428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39646 (local address 127.25.137.252:36487)
0808 02:47:06.654608 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.654613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.654982 (+ 369us) server_negotiation.cc:407] Connection header received
0808 02:47:06.655273 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.655279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.655368 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.655441 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.657738 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.658893 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.660990 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.661707 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.661799 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.666041 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.666048 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.666053 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.666093 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.670024 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.671086 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.671093 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.671097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.671272 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.671783 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.671789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.671792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.672209 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.672440 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.673154 (+ 714us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.673413 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:47:06.674257 15007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } } }
I20260808 02:47:06.674685 15007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:47:06.676630 14984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:47:06.681602 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.655419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57166 (local address 127.25.137.253:37529)
0808 02:47:06.655581 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.655586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.666509 (+ 10923us) server_negotiation.cc:407] Connection header received
0808 02:47:06.666650 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.666655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.666733 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.666838 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.668517 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.669646 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.671581 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.672201 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.672293 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.675710 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.675716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.675721 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.675763 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.678617 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.679516 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.679520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.679522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.679649 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.680068 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.680074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.680077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.680470 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.680697 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.681052 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.681279 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 02:47:06.686766 15021 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:47:06.686939 15021 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:47:06.683702 14881 master_runner.cc:186] Error getting master registration for 127.25.137.254:44229: OK, instance_id { permanent_uuid: "51750762300744db8fa07f4a05b99e19" instance_seqno: 1786157225563569 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:47:06.692229 14881 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:47:06.704066 14765 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } attempt: 4
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157225, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
I20260808 02:47:06.719453 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.704715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39660)
0808 02:47:06.704925 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.704945 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:06.705093 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:06.705474 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:06.705477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:06.705914 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:06.706101 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.706111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.707588 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.707592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.708611 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.708628 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.710025 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.710031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.710242 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.710907 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:06.710926 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:06.713428 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:06.716725 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.716731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.716745 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.717223 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.717632 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.717638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.717643 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.717799 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.718400 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:06.718406 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:06.718561 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 02:47:06.718693 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 02:47:06.718996 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 02:47:06.719501 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.704783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.25.137.252:36487)
0808 02:47:06.705045 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.705051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.705076 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:47:06.705222 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.705227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.705334 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.705446 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.706253 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.707447 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.708835 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.709605 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.709784 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.713615 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.713620 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.713623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.713658 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.716584 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.717355 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.717361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.717364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.717515 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.717923 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.717926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.717928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.718258 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.718411 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.718848 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.719082 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 02:47:06.724862 14765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19: Bootstrap starting.
I20260808 02:47:06.728144 14765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:06.729348 14765 log.cc:824] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:06.732312 14765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19: No bootstrap required, opened a new log
I20260808 02:47:06.740628 14765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } }
I20260808 02:47:06.740944 14765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:06.741084 14765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51750762300744db8fa07f4a05b99e19, State: Initialized, Role: FOLLOWER
I20260808 02:47:06.741689 14765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } }
I20260808 02:47:06.742614 15025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } } }
I20260808 02:47:06.743096 15025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:47:06.744078 14765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:47:06.763849 15049 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:47:06.764078 15049 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:47:06.756815 14651 master_runner.cc:186] Error getting master registration for 127.25.137.253:37529: OK, instance_id { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" instance_seqno: 1786157226066774 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:47:06.766274 14651 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:47:06.791847 14872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } attempt: 5
Aug 08 02:47:06 dist-test-slave-k5rr krb5kdc[14627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157226, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
W20260808 02:47:06.802940 14682 tablet.cc:2369] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:47:06.810874 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.792516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:36487 (local address 127.0.0.1:39670)
0808 02:47:06.792799 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:06.792820 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:06.792935 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:06.793399 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:06.793405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:06.793934 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:06.794149 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.794163 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.795653 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.795659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.797107 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:06.797124 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.798461 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.798469 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:06.798708 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.800569 (+ 1861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:06.800593 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:06.803576 (+ 2983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:06.807971 (+ 4395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.807976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.807980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.808532 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.808995 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:06.809000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:06.809026 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:06.809192 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:06.809886 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:06.809891 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:06.810063 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 02:47:06.810215 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 02:47:06.810500 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260808 02:47:06.811412 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:06.792586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39670 (local address 127.25.137.252:36487)
0808 02:47:06.792831 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:06.792838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:06.792868 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:47:06.793144 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:06.793150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:06.793243 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:06.793362 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:06.794325 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.795486 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.797268 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:06.798059 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:06.798210 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:06.803813 (+ 5603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:06.803827 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:06.803832 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:06.803878 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:06.807814 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.808686 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.808692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.808696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.808873 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:06.809328 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:06.809334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:06.809338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:06.809751 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:06.809938 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:06.810380 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 02:47:06.810651 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260808 02:47:06.812379 14912 tablet.cc:2369] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:47:06.817898 14872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c: Bootstrap starting.
I20260808 02:47:06.821913 14872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:06.823179 14872 log.cc:824] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:06.826493 14872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c: No bootstrap required, opened a new log
I20260808 02:47:06.837131 14872 raft_consensus.cc:348] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } }
I20260808 02:47:06.837532 14872 raft_consensus.cc:374] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:06.837646 14872 raft_consensus.cc:729] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e08f948acfe447878e6dd0df8d3f206c, State: Initialized, Role: FOLLOWER
I20260808 02:47:06.838279 14872 consensus_queue.cc:260] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } }
I20260808 02:47:06.841902 15053 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } } }
I20260808 02:47:06.842413 15053 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:47:06.843146 14872 sys_catalog.cc:564] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:47:06.862500 14774 master_runner.cc:263] No leader master found from master e08f948acfe447878e6dd0df8d3f206c
I20260808 02:47:06.862671 14774 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 02:47:06.863103 15068 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:47:06.863297 15068 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:47:06.890597 15053 raft_consensus.cc:482] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:47:06.890978 15053 raft_consensus.cc:504] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } }
I20260808 02:47:06.892771 15053 leader_election.cc:288] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51750762300744db8fa07f4a05b99e19 (127.25.137.254:44229), 03f7fa69e79c4247a8e4f3aa4a471f01 (127.25.137.252:36487)
I20260808 02:47:06.894037 14721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e08f948acfe447878e6dd0df8d3f206c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51750762300744db8fa07f4a05b99e19" is_pre_election: true
I20260808 02:47:06.894707 14721 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e08f948acfe447878e6dd0df8d3f206c in term 0.
I20260808 02:47:06.895624 14803 leader_election.cc:302] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51750762300744db8fa07f4a05b99e19, e08f948acfe447878e6dd0df8d3f206c; no voters:
I20260808 02:47:06.896885 14949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e08f948acfe447878e6dd0df8d3f206c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" is_pre_election: true
I20260808 02:47:06.897607 14949 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e08f948acfe447878e6dd0df8d3f206c in term 0.
I20260808 02:47:06.902976 15053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:47:06.903168 15053 raft_consensus.cc:482] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:47:06.903287 15053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:06.906982 15053 raft_consensus.cc:504] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } }
I20260808 02:47:06.908313 14721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e08f948acfe447878e6dd0df8d3f206c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51750762300744db8fa07f4a05b99e19"
I20260808 02:47:06.908571 14721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:06.912164 14721 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e08f948acfe447878e6dd0df8d3f206c in term 1.
I20260808 02:47:06.912741 14803 leader_election.cc:302] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51750762300744db8fa07f4a05b99e19, e08f948acfe447878e6dd0df8d3f206c; no voters:
I20260808 02:47:06.918012 14949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e08f948acfe447878e6dd0df8d3f206c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01"
I20260808 02:47:06.918287 14949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:06.922139 14949 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e08f948acfe447878e6dd0df8d3f206c in term 1.
I20260808 02:47:06.923244 15053 leader_election.cc:288] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [CANDIDATE]: Term 1 election: Requested vote from peers 51750762300744db8fa07f4a05b99e19 (127.25.137.254:44229), 03f7fa69e79c4247a8e4f3aa4a471f01 (127.25.137.252:36487)
I20260808 02:47:06.923738 15053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:47:06.928301 15053 raft_consensus.cc:686] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [term 1 LEADER]: Becoming Leader. State: Replica: e08f948acfe447878e6dd0df8d3f206c, State: Running, Role: LEADER
I20260808 02:47:06.929520 15053 consensus_queue.cc:237] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } }
I20260808 02:47:06.944684 15074 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e08f948acfe447878e6dd0df8d3f206c. Latest consensus state: current_term: 1 leader_uuid: "e08f948acfe447878e6dd0df8d3f206c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } } }
I20260808 02:47:06.945968 15074 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:06.957623 15080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:47:06.960189 15080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:47:06.982625 14949 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [term 1 FOLLOWER]: Refusing update from remote peer e08f948acfe447878e6dd0df8d3f206c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:47:06.983819 15074 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [LEADER]: Connected to new peer: Peer: permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:47:06.991616 14721 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [term 1 FOLLOWER]: Refusing update from remote peer e08f948acfe447878e6dd0df8d3f206c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:47:06.992604 15074 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [LEADER]: Connected to new peer: Peer: permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:47:07.015350 15007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e08f948acfe447878e6dd0df8d3f206c. Latest consensus state: current_term: 1 leader_uuid: "e08f948acfe447878e6dd0df8d3f206c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } } }
I20260808 02:47:07.015688 15007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:47:07.022596 15082 mvcc.cc:205] Tried to move back new op lower bound from 7316100001707913216 to 7316100001516474368. Current Snapshot: MvccSnapshot[applied={T|T < 7316100001707913216}]
I20260808 02:47:07.029278 15025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e08f948acfe447878e6dd0df8d3f206c. Latest consensus state: current_term: 1 leader_uuid: "e08f948acfe447878e6dd0df8d3f206c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } } }
I20260808 02:47:07.029598 15025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:47:07.033644 15007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e08f948acfe447878e6dd0df8d3f206c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } } }
I20260808 02:47:07.033949 15007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:47:07.045763 15025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e08f948acfe447878e6dd0df8d3f206c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } } }
I20260808 02:47:07.046142 15025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:47:07.050719 15074 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e08f948acfe447878e6dd0df8d3f206c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } } }
I20260808 02:47:07.051045 15074 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:07.051584 15074 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e08f948acfe447878e6dd0df8d3f206c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51750762300744db8fa07f4a05b99e19" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 44229 } } peers { permanent_uuid: "e08f948acfe447878e6dd0df8d3f206c" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 37529 } } peers { permanent_uuid: "03f7fa69e79c4247a8e4f3aa4a471f01" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 36487 } } }
I20260808 02:47:07.051831 15074 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:07.052698 15080 catalog_manager.cc:1347] Generated new cluster ID: 2a527744d18a4f2f94c5ee6f7797d533
I20260808 02:47:07.052822 15080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:47:07.081099 15080 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:47:07.083020 15080 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:47:07.113049 15080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e08f948acfe447878e6dd0df8d3f206c: Generated new TSK 0
I20260808 02:47:07.114217 15080 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:47:07.689302 15021 catalog_manager.cc:1259] Loaded cluster ID: 2a527744d18a4f2f94c5ee6f7797d533
I20260808 02:47:07.689426 15021 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01: loading cluster ID for follower catalog manager: success
I20260808 02:47:07.692096 15021 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01: acquiring CA information for follower catalog manager: success
I20260808 02:47:07.694052 15021 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 03f7fa69e79c4247a8e4f3aa4a471f01: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:47:07.766243 15049 catalog_manager.cc:1259] Loaded cluster ID: 2a527744d18a4f2f94c5ee6f7797d533
I20260808 02:47:07.766355 15049 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19: loading cluster ID for follower catalog manager: success
I20260808 02:47:07.768771 15049 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19: acquiring CA information for follower catalog manager: success
I20260808 02:47:07.770589 15049 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 51750762300744db8fa07f4a05b99e19: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:47:08.253387 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 14651
I20260808 02:47:08.254807 14754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:08.364459 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 14651
I20260808 02:47:08.393038 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 14774
I20260808 02:47:08.394251 14864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:08.503134 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 14774
I20260808 02:47:08.528796 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 14881
I20260808 02:47:08.529922 14979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:08.637214 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 14881
2026-08-08T02:47:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5066 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:47:08 dist-test-slave-k5rr krb5kdc[15109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:47:08 dist-test-slave-k5rr krb5kdc[15109](info): set up 2 sockets
Aug 08 02:47:08 dist-test-slave-k5rr krb5kdc[15109](info): commencing operation
krb5kdc: starting...
W20260808 02:47:09.864380 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:47:10 dist-test-slave-k5rr krb5kdc[15109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:47:10.146611 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:42461
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:46631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:42461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:10.488045 15131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:10.488343 15131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:10.488442 15131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:10.495779 15131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:47:10.495894 15131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:10.495962 15131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:10.496034 15131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:47:10.496088 15131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:47:10.506094 15131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:46631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:42461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:42461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:10.507359 15131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:10.508694 15131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:10.520957 15138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:10.520992 15139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:10.521032 15141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:10.522090 15131 server_base.cc:1034] running on GCE node
I20260808 02:47:10.522879 15131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:10.524315 15131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:10.525528 15131 hybrid_clock.cc:648] HybridClock initialized: now 1786157230525477 us; error 33 us; skew 500 ppm
I20260808 02:47:10.525876 15131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:10 dist-test-slave-k5rr krb5kdc[15109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157230, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:10.532780 15131 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:47:10.534461 15131 webserver.cc:460] Webserver started at http://127.25.137.254:33793/ using document root <none> and password file <none>
I20260808 02:47:10.535176 15131 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:10.535298 15131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:10.535604 15131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:10.538662 15131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "5bcd169d0c1d4a6298c9d659540735c8"
format_stamp: "Formatted at 2026-08-08 02:47:10 on dist-test-slave-k5rr"
server_key: "5106af649e9eb0873e8b0e9282f49bc0"
server_key_iv: "1adc59fdc7e5c49ab4dc0ab02b57e064"
server_key_version: "encryptionkey@0"
I20260808 02:47:10.539419 15131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "5bcd169d0c1d4a6298c9d659540735c8"
format_stamp: "Formatted at 2026-08-08 02:47:10 on dist-test-slave-k5rr"
server_key: "5106af649e9eb0873e8b0e9282f49bc0"
server_key_iv: "1adc59fdc7e5c49ab4dc0ab02b57e064"
server_key_version: "encryptionkey@0"
I20260808 02:47:10.544492 15131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:47:10.547865 15153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:10.549203 15131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:47:10.549364 15131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "5bcd169d0c1d4a6298c9d659540735c8"
format_stamp: "Formatted at 2026-08-08 02:47:10 on dist-test-slave-k5rr"
server_key: "5106af649e9eb0873e8b0e9282f49bc0"
server_key_iv: "1adc59fdc7e5c49ab4dc0ab02b57e064"
server_key_version: "encryptionkey@0"
I20260808 02:47:10.549561 15131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:10.576607 15131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:10.580705 15131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:10.580986 15131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:10.599130 15131 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:42461
I20260808 02:47:10.599135 15231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:42461 every 8 connection(s)
I20260808 02:47:10.600534 15131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:47:10.604219 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 15131
I20260808 02:47:10.604369 15232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:10.604420 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:47:10.605139 26151 external_mini_cluster.cc:1428] Setting key 7b2c854eb4b49aad14a124b8a8deb1ea
I20260808 02:47:10.619279 15232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8: Bootstrap starting.
Aug 08 02:47:10 dist-test-slave-k5rr krb5kdc[15109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:10.623420 15232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:10.624861 15232 log.cc:824] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:10.628491 15232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8: No bootstrap required, opened a new log
I20260808 02:47:10.630261 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:10.606968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53776 (local address 127.25.137.254:42461)
0808 02:47:10.607896 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:10.607912 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:10.607966 (+ 54us) server_negotiation.cc:407] Connection header received
0808 02:47:10.609776 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:10.609809 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:10.610324 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:10.610783 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:10.612138 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:10.614179 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:10.615591 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:10.616759 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:10.616971 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:10.621257 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:10.621273 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:10.621296 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:10.621346 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:10.625439 (+ 4093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:10.626399 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:10.626408 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:10.626417 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:10.626645 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:10.627059 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:10.627064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:10.627068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:10.627783 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:10.628069 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:10.628413 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 02:47:10.628863 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":352,"threads_started":1}
I20260808 02:47:10.634114 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:47:10.638350 15232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcd169d0c1d4a6298c9d659540735c8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42461 } }
I20260808 02:47:10.638713 15232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:10.638806 15232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bcd169d0c1d4a6298c9d659540735c8, State: Initialized, Role: FOLLOWER
I20260808 02:47:10.639320 15232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcd169d0c1d4a6298c9d659540735c8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42461 } }
I20260808 02:47:10.639549 15232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:47:10.639654 15232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:47:10.639786 15232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:10.642025 15232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcd169d0c1d4a6298c9d659540735c8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42461 } }
I20260808 02:47:10.642503 15232 leader_election.cc:302] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bcd169d0c1d4a6298c9d659540735c8; no voters:
I20260808 02:47:10.643054 15232 leader_election.cc:288] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:47:10.643172 15238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:47:10.643926 15238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [term 1 LEADER]: Becoming Leader. State: Replica: 5bcd169d0c1d4a6298c9d659540735c8, State: Running, Role: LEADER
I20260808 02:47:10.644444 15238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcd169d0c1d4a6298c9d659540735c8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42461 } }
I20260808 02:47:10.645165 15232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:47:10.647291 15241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bcd169d0c1d4a6298c9d659540735c8. Latest consensus state: current_term: 1 leader_uuid: "5bcd169d0c1d4a6298c9d659540735c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcd169d0c1d4a6298c9d659540735c8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42461 } } }
I20260808 02:47:10.647189 15239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bcd169d0c1d4a6298c9d659540735c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bcd169d0c1d4a6298c9d659540735c8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42461 } } }
I20260808 02:47:10.648417 15241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:10.648438 15239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:10.655791 15254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:47:10.660244 15254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:47:10.668354 15254 catalog_manager.cc:1347] Generated new cluster ID: 1e9d306597ed4b0cacbef801f015538c
I20260808 02:47:10.668470 15254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:47:10.682652 15254 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:47:10.684527 15254 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:47:10.700459 15254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bcd169d0c1d4a6298c9d659540735c8: Generated new TSK 0
I20260808 02:47:10.701375 15254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:47:10.747162 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 15131
I20260808 02:47:10.748524 15226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:10.844893 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 15131
2026-08-08T02:47:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2206 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:47:10 dist-test-slave-k5rr krb5kdc[15274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:47:10 dist-test-slave-k5rr krb5kdc[15274](info): set up 2 sockets
Aug 08 02:47:10 dist-test-slave-k5rr krb5kdc[15274](info): commencing operation
krb5kdc: starting...
W20260808 02:47:12.074846 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:47:12 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:47:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:47:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:47:12.360405 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:43729
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:43729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:12.691279 15295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:12.691587 15295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:12.691706 15295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:12.698871 15295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:47:12.698985 15295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:12.699038 15295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:12.699116 15295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:47:12.699165 15295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:47:12.709079 15295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:43729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:43729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:12.710391 15295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:12.711705 15295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:12.723233 15307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:12.723315 15302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:12.723289 15304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:12.723934 15295 server_base.cc:1034] running on GCE node
I20260808 02:47:12.724911 15295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:12.726557 15295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:12.727777 15295 hybrid_clock.cc:648] HybridClock initialized: now 1786157232727726 us; error 40 us; skew 500 ppm
I20260808 02:47:12.728112 15295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:12 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157232, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:12.731187 15295 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:47:12.732656 15295 webserver.cc:460] Webserver started at http://127.25.137.254:43819/ using document root <none> and password file <none>
I20260808 02:47:12.733355 15295 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:12.733469 15295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:12.733749 15295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:12.736605 15295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "fb3fb0cb9614444ca57c3d0959d15b50"
format_stamp: "Formatted at 2026-08-08 02:47:12 on dist-test-slave-k5rr"
server_key: "1c64a7e11a492ec312b743bd521df961"
server_key_iv: "d7a7138aeff53c6eddb52d012068bfc9"
server_key_version: "encryptionkey@0"
I20260808 02:47:12.737377 15295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "fb3fb0cb9614444ca57c3d0959d15b50"
format_stamp: "Formatted at 2026-08-08 02:47:12 on dist-test-slave-k5rr"
server_key: "1c64a7e11a492ec312b743bd521df961"
server_key_iv: "d7a7138aeff53c6eddb52d012068bfc9"
server_key_version: "encryptionkey@0"
I20260808 02:47:12.742206 15295 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:47:12.745668 15316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:12.746999 15295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:47:12.747153 15295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "fb3fb0cb9614444ca57c3d0959d15b50"
format_stamp: "Formatted at 2026-08-08 02:47:12 on dist-test-slave-k5rr"
server_key: "1c64a7e11a492ec312b743bd521df961"
server_key_iv: "d7a7138aeff53c6eddb52d012068bfc9"
server_key_version: "encryptionkey@0"
I20260808 02:47:12.747335 15295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:12.761943 15295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:12.765877 15295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:12.766135 15295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:12.784209 15295 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:43729
I20260808 02:47:12.784260 15393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:43729 every 8 connection(s)
I20260808 02:47:12.785642 15295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:47:12.786417 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 15295
I20260808 02:47:12.786653 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:47:12.787494 26151 external_mini_cluster.cc:1428] Setting key 364e8dcb306304e9389d69977837d34b
I20260808 02:47:12.789870 15394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:12.800697 15394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Bootstrap starting.
Aug 08 02:47:12 dist-test-slave-k5rr krb5kdc[15274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:12.803732 15394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:12.804771 15394 log.cc:824] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:12.807766 15394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: No bootstrap required, opened a new log
I20260808 02:47:12.810940 15402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:12.789533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52396 (local address 127.25.137.254:43729)
0808 02:47:12.790363 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:12.790376 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:12.790420 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:47:12.792192 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:12.792215 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:12.792630 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:12.793116 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:12.794745 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:12.796134 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:12.797583 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:12.798374 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:12.798580 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:12.802986 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:12.802994 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:12.803007 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:12.803048 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:12.806657 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:12.807363 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:12.807369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:12.807375 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:12.807569 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:12.807994 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:12.807999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:12.808002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:12.808567 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:12.808917 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:12.809410 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 02:47:12.809861 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":357,"threads_started":1}
I20260808 02:47:12.815960 15394 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:12.816293 15394 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:12.816394 15394 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb3fb0cb9614444ca57c3d0959d15b50, State: Initialized, Role: FOLLOWER
I20260808 02:47:12.816928 15394 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:12.817198 15394 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:47:12.817340 15394 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:47:12.817497 15394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:12.819837 15394 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:12.820358 15394 leader_election.cc:302] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb3fb0cb9614444ca57c3d0959d15b50; no voters:
I20260808 02:47:12.821089 15394 leader_election.cc:288] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:47:12.821230 15404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:47:12.822046 15404 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 1 LEADER]: Becoming Leader. State: Replica: fb3fb0cb9614444ca57c3d0959d15b50, State: Running, Role: LEADER
I20260808 02:47:12.822537 15404 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:12.823114 15394 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:47:12.825176 15405 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } } }
I20260808 02:47:12.825543 15405 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:12.826113 15409 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb3fb0cb9614444ca57c3d0959d15b50. Latest consensus state: current_term: 1 leader_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } } }
I20260808 02:47:12.829717 15409 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:12.830308 15415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:47:12.835093 15415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:47:12.844188 15415 catalog_manager.cc:1347] Generated new cluster ID: de8e7cb8501e45d0a13dcb0f592c41f1
I20260808 02:47:12.844282 15415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:47:12.870177 15415 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:47:12.871543 15415 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:47:12.878631 15415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Generated new TSK 0
I20260808 02:47:12.879603 15415 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:47:12.952853 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:43729
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:13.289492 15434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:13.289816 15434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:13.289978 15434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:13.297684 15434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:13.297873 15434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:13.298030 15434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:47:13.307834 15434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:43729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:13.309129 15434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:13.310437 15434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:13.322939 15442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:13.322871 15447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:13.322901 15443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:13.324059 15434 server_base.cc:1034] running on GCE node
I20260808 02:47:13.324762 15434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:13.325712 15434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:13.326926 15434 hybrid_clock.cc:648] HybridClock initialized: now 1786157233326874 us; error 44 us; skew 500 ppm
I20260808 02:47:13.327231 15434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:13 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157233, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:13.330482 15434 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:47:13.331900 15434 webserver.cc:460] Webserver started at http://127.25.137.193:40805/ using document root <none> and password file <none>
I20260808 02:47:13.332554 15434 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:13.332679 15434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:13.332960 15434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:13.335884 15434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "49b1e360420644508f80772cd56ac08b"
format_stamp: "Formatted at 2026-08-08 02:47:13 on dist-test-slave-k5rr"
server_key: "010819d033aa166e539a440acbceb1b3"
server_key_iv: "e1c6078efe5fd2d823e49194fb7531ff"
server_key_version: "encryptionkey@0"
I20260808 02:47:13.336732 15434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "49b1e360420644508f80772cd56ac08b"
format_stamp: "Formatted at 2026-08-08 02:47:13 on dist-test-slave-k5rr"
server_key: "010819d033aa166e539a440acbceb1b3"
server_key_iv: "e1c6078efe5fd2d823e49194fb7531ff"
server_key_version: "encryptionkey@0"
I20260808 02:47:13.341881 15434 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:47:13.345367 15460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:13.346674 15434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 02:47:13.346815 15434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "49b1e360420644508f80772cd56ac08b"
format_stamp: "Formatted at 2026-08-08 02:47:13 on dist-test-slave-k5rr"
server_key: "010819d033aa166e539a440acbceb1b3"
server_key_iv: "e1c6078efe5fd2d823e49194fb7531ff"
server_key_version: "encryptionkey@0"
I20260808 02:47:13.347003 15434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:13.361608 15434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:13.365658 15434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:13.365924 15434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:13.366873 15434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:13.368242 15434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:13.368322 15434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:13.368434 15434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:13.368492 15434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:13.399569 15434 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:33067
I20260808 02:47:13.399613 15631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:33067 every 8 connection(s)
I20260808 02:47:13.400897 15434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:47:13.402954 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 15434
I20260808 02:47:13.403185 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:47:13.403950 26151 external_mini_cluster.cc:1428] Setting key 2b2233fa19803c4479b06e20e1e49b99
Aug 08 02:47:13 dist-test-slave-k5rr krb5kdc[15274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157233, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:13.420660 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:13.403840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:58245 (local address 127.25.137.254:43729)
0808 02:47:13.404396 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:13.404402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:13.405429 (+ 1027us) server_negotiation.cc:407] Connection header received
0808 02:47:13.407267 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:13.407274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:13.407360 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:13.407454 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:13.408964 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:13.409733 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:13.411249 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:13.411652 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:13.411775 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:13.415104 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:13.415109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:13.415112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:13.415145 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:13.417641 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:13.418343 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:13.418348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:13.418350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:13.418474 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:13.418893 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:13.418897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:13.418901 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:13.419167 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:13.419342 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:13.420052 (+ 710us) server_negotiation.cc:299] Negotiation successful
0808 02:47:13.420284 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":263,"threads_started":1}
I20260808 02:47:13.421692 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:13.404388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:58245)
0808 02:47:13.405263 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:13.405309 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:13.407006 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:13.407708 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:13.407716 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:13.408222 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:13.408782 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:13.408812 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:13.409888 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:13.409892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:13.411100 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:13.411112 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:13.411885 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:13.411893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:13.412107 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:13.412720 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:13.412742 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:13.414896 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:13.417790 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:13.417808 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:13.417820 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:13.418220 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:13.418579 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:13.418585 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:13.418588 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:13.418777 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:13.419295 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:13.419303 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:13.419647 (+ 344us) client_negotiation.cc:769] Sending connection context
0808 02:47:13.419904 (+ 257us) client_negotiation.cc:240] Negotiation successful
0808 02:47:13.420438 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":387,"threads_started":1}
I20260808 02:47:13.423281 15632 heartbeater.cc:347] Connected to a master server at 127.25.137.254:43729
I20260808 02:47:13.423652 15632 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:13.424664 15632 heartbeater.cc:511] Master 127.25.137.254:43729 requested a full tablet report, sending...
I20260808 02:47:13.427124 15341 ts_manager.cc:194] Registered new tserver with Master: 49b1e360420644508f80772cd56ac08b (127.25.137.193:33067)
I20260808 02:47:13.428577 15341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:58245
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:47:13.467571 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:43729
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:13.794821 15640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:13.795077 15640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:13.795195 15640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:13.802464 15640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:13.802575 15640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:13.802727 15640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:47:13.812253 15640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:43729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:13.813550 15640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:13.814806 15640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:13.827327 15647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:13.827286 15650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:13.827358 15648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:13.828444 15640 server_base.cc:1034] running on GCE node
I20260808 02:47:13.829108 15640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:13.830065 15640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:13.831324 15640 hybrid_clock.cc:648] HybridClock initialized: now 1786157233831250 us; error 50 us; skew 500 ppm
I20260808 02:47:13.831688 15640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:13 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157233, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:13.834894 15640 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:47:13.836239 15640 webserver.cc:460] Webserver started at http://127.25.137.194:36553/ using document root <none> and password file <none>
I20260808 02:47:13.836920 15640 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:13.837059 15640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:13.837337 15640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:13.840205 15640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "8d9d6454766345c986b8bc519d8432e5"
format_stamp: "Formatted at 2026-08-08 02:47:13 on dist-test-slave-k5rr"
server_key: "1aca603eef7e19a51969830fe22bc225"
server_key_iv: "bbb64766eb54a39d14220df977fb61cb"
server_key_version: "encryptionkey@0"
I20260808 02:47:13.840981 15640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "8d9d6454766345c986b8bc519d8432e5"
format_stamp: "Formatted at 2026-08-08 02:47:13 on dist-test-slave-k5rr"
server_key: "1aca603eef7e19a51969830fe22bc225"
server_key_iv: "bbb64766eb54a39d14220df977fb61cb"
server_key_version: "encryptionkey@0"
I20260808 02:47:13.846036 15640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 02:47:13.849578 15662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:13.851186 15640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:47:13.851428 15640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "8d9d6454766345c986b8bc519d8432e5"
format_stamp: "Formatted at 2026-08-08 02:47:13 on dist-test-slave-k5rr"
server_key: "1aca603eef7e19a51969830fe22bc225"
server_key_iv: "bbb64766eb54a39d14220df977fb61cb"
server_key_version: "encryptionkey@0"
I20260808 02:47:13.851666 15640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:13.910063 15640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:13.914351 15640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:13.914631 15640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:13.915652 15640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:13.917090 15640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:13.917171 15640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:13.917300 15640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:13.917357 15640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:13.948158 15640 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:36329
I20260808 02:47:13.948197 15838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:36329 every 8 connection(s)
I20260808 02:47:13.949470 15640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:47:13.958936 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 15640
I20260808 02:47:13.959105 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:47:13.959553 26151 external_mini_cluster.cc:1428] Setting key 30e04a14c554338f3343a925c801e80f
Aug 08 02:47:13 dist-test-slave-k5rr krb5kdc[15274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157233, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:13.970014 15843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:13.952221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:41245 (local address 127.25.137.254:43729)
0808 02:47:13.952671 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:13.952676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:13.953736 (+ 1060us) server_negotiation.cc:407] Connection header received
0808 02:47:13.955572 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:13.955577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:13.955665 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:13.955749 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:13.957356 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:13.958106 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:13.959757 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:13.960270 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:13.960405 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:13.964419 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:13.964424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:13.964428 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:13.964462 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:13.966990 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:13.967817 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:13.967822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:13.967824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:13.967962 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:13.968393 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:13.968396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:13.968398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:13.968660 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:13.968777 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:13.969575 (+ 798us) server_negotiation.cc:299] Negotiation successful
0808 02:47:13.969745 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":214,"threads_started":1}
I20260808 02:47:13.971276 15846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:13.952819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:41245)
0808 02:47:13.953576 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:13.953614 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:13.955303 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:13.955993 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:13.956001 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:13.956493 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:13.957153 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:13.957174 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:13.958257 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:13.958261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:13.959611 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:13.959628 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:13.960520 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:13.960528 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:13.960743 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:13.961436 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:13.961468 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:13.964169 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:13.967135 (+ 2966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:13.967143 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:13.967158 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:13.967675 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:13.968062 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:13.968066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:13.968070 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:13.968302 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:13.968781 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:13.968788 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:13.969210 (+ 422us) client_negotiation.cc:769] Sending connection context
0808 02:47:13.969490 (+ 280us) client_negotiation.cc:240] Negotiation successful
0808 02:47:13.969987 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":308,"threads_started":1}
I20260808 02:47:13.972653 15839 heartbeater.cc:347] Connected to a master server at 127.25.137.254:43729
I20260808 02:47:13.972946 15839 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:13.973695 15839 heartbeater.cc:511] Master 127.25.137.254:43729 requested a full tablet report, sending...
I20260808 02:47:13.975307 15341 ts_manager.cc:194] Registered new tserver with Master: 8d9d6454766345c986b8bc519d8432e5 (127.25.137.194:36329)
I20260808 02:47:13.976111 15341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:41245
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:47:14.020201 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:43729
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:14.350462 15850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:14.350754 15850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:14.350894 15850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:14.358266 15850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:14.358381 15850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:14.358531 15850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:47:14.368036 15850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:43729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:14.369292 15850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:14.370559 15850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:14.382723 15860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:14.382843 15861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:14.382833 15866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:14.383697 15850 server_base.cc:1034] running on GCE node
I20260808 02:47:14.384320 15850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:14.385259 15850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:14.386475 15850 hybrid_clock.cc:648] HybridClock initialized: now 1786157234386419 us; error 39 us; skew 500 ppm
I20260808 02:47:14.386817 15850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:14 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157234, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:14.389822 15850 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:47:14.391160 15850 webserver.cc:460] Webserver started at http://127.25.137.195:39319/ using document root <none> and password file <none>
I20260808 02:47:14.391811 15850 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:14.391930 15850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:14.392215 15850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:14.395066 15850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "3c8d38c776aa49099ff618715fa41bb8"
format_stamp: "Formatted at 2026-08-08 02:47:14 on dist-test-slave-k5rr"
server_key: "c2cb30737da16a77456262b3d7ec85c1"
server_key_iv: "da2e25ec69ebb5c21b8e38a740899c2d"
server_key_version: "encryptionkey@0"
I20260808 02:47:14.395815 15850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "3c8d38c776aa49099ff618715fa41bb8"
format_stamp: "Formatted at 2026-08-08 02:47:14 on dist-test-slave-k5rr"
server_key: "c2cb30737da16a77456262b3d7ec85c1"
server_key_iv: "da2e25ec69ebb5c21b8e38a740899c2d"
server_key_version: "encryptionkey@0"
I20260808 02:47:14.401121 15850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:47:14.404614 15875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:14.405987 15850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:47:14.406140 15850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "3c8d38c776aa49099ff618715fa41bb8"
format_stamp: "Formatted at 2026-08-08 02:47:14 on dist-test-slave-k5rr"
server_key: "c2cb30737da16a77456262b3d7ec85c1"
server_key_iv: "da2e25ec69ebb5c21b8e38a740899c2d"
server_key_version: "encryptionkey@0"
I20260808 02:47:14.406316 15850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:14.428004 15850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:14.432125 15632 heartbeater.cc:503] Master 127.25.137.254:43729 was elected leader, sending a full tablet report...
I20260808 02:47:14.432245 15850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:14.432564 15850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:14.433787 15850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:14.435222 15850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:14.435300 15850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:14.435416 15850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:14.435465 15850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:14.466648 15850 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:38885
I20260808 02:47:14.466692 16065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:38885 every 8 connection(s)
I20260808 02:47:14.467922 15850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:47:14.477845 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 15850
I20260808 02:47:14.478025 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:47:14.478447 26151 external_mini_cluster.cc:1428] Setting key e8e11a59578b405d6f484899fdc6afeb
Aug 08 02:47:14 dist-test-slave-k5rr krb5kdc[15274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157234, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:14.487816 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.470523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:52581 (local address 127.25.137.254:43729)
0808 02:47:14.471003 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.471009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.471942 (+ 933us) server_negotiation.cc:407] Connection header received
0808 02:47:14.473894 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.473898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.473967 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.474062 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:14.475520 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.476285 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.477883 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.478377 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.478490 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.482402 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:14.482406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:14.482410 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:14.482445 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:14.485125 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:14.485822 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:14.485826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:14.485828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:14.485946 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:14.486286 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:14.486289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:14.486291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:14.486545 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:14.486694 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.487374 (+ 680us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.487538 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":196,"threads_started":1}
I20260808 02:47:14.488742 16069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.471059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:52581)
0808 02:47:14.471787 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:14.471829 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:14.473654 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:14.474247 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:14.474280 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:14.474788 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:14.475355 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.475373 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.476437 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.476441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.477723 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.477736 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.478610 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.478617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.478842 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.479676 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:14.479701 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:14.482208 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:14.485261 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:14.485268 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:14.485280 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:14.485690 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:14.486028 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:14.486031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:14.486033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:14.486204 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:14.486666 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:14.486674 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:14.487030 (+ 356us) client_negotiation.cc:769] Sending connection context
0808 02:47:14.487244 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 02:47:14.487634 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":273,"threads_started":1}
I20260808 02:47:14.490088 16066 heartbeater.cc:347] Connected to a master server at 127.25.137.254:43729
I20260808 02:47:14.490384 16066 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:14.491092 16066 heartbeater.cc:511] Master 127.25.137.254:43729 requested a full tablet report, sending...
I20260808 02:47:14.492746 15341 ts_manager.cc:194] Registered new tserver with Master: 3c8d38c776aa49099ff618715fa41bb8 (127.25.137.195:38885)
I20260808 02:47:14.493661 15341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:52581
I20260808 02:47:14.494189 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:47:14.511126 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.498613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52402 (local address 127.25.137.254:43729)
0808 02:47:14.498796 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.498802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.499146 (+ 344us) server_negotiation.cc:407] Connection header received
0808 02:47:14.499341 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.499346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.499407 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.499487 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:14.500893 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.501672 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.502998 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.503396 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.503503 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.505893 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:14.505898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:14.505902 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:14.505934 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:14.508284 (+ 2350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:14.509169 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:14.509180 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:14.509184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:14.509340 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:14.509777 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:14.509783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:14.509787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:14.510128 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:14.510376 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.510635 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.510859 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:47:14.515239 15341 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:47:14.522588 15341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:47:14.545131 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.535435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54996 (local address 127.25.137.193:33067)
0808 02:47:14.536868 (+ 1433us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.536875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.536902 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:47:14.536987 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.536993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.537237 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.537578 (+ 341us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:14.538417 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.539902 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.542915 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.543963 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.544117 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.544246 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.544621 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.544788 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1242,"thread_start_us":1368,"threads_started":1}
I20260808 02:47:14.545346 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.534806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:33067 (local address 127.0.0.1:54996)
0808 02:47:14.536210 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:14.536227 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:14.536335 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:14.537459 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:14.537464 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:14.537494 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:14.538185 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.538198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.540030 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.540034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.542724 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.542740 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.544160 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.544166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.544359 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.544438 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 02:47:14.544561 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 02:47:14.544759 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1279,"mutex_wait_us":356,"thread_start_us":1155,"threads_started":1}
I20260808 02:47:14.546985 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.534703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:38885 (local address 127.0.0.1:46330)
0808 02:47:14.535441 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:14.535485 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:14.535651 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:14.536977 (+ 1326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:14.536984 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:14.537043 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:14.537580 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.537594 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.539972 (+ 2378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.539977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.542914 (+ 2937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.542926 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.545965 (+ 3039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.545972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.546140 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.546172 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:47:14.546270 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 02:47:14.546405 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":502,"thread_start_us":367,"threads_started":1}
I20260808 02:47:14.548759 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.536941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:36329 (local address 127.0.0.1:43198)
0808 02:47:14.541469 (+ 4528us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:14.541483 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:14.541609 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:14.542078 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:14.542083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:14.542107 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:14.542571 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.542585 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.544300 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.544305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.546413 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.546428 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.548185 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.548189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.548327 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.548349 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:47:14.548426 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 02:47:14.548507 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4399,"mutex_wait_us":21,"thread_start_us":4220,"threads_started":1}
I20260808 02:47:14.548955 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.537380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.25.137.194:36329)
0808 02:47:14.541066 (+ 3686us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.541074 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.541583 (+ 509us) server_negotiation.cc:407] Connection header received
0808 02:47:14.541751 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.541757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.541908 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.542063 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:14.542777 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.544143 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.546616 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.547972 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.548136 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.548279 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.548494 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.548648 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3504,"thread_start_us":3360,"threads_started":1}
I20260808 02:47:14.549542 15542 tablet_service.cc:1467] Processing CreateTablet for tablet 2c04dfee972a4f61a4e95ce95051d321 (DEFAULT_TABLE table=test-table [id=6cd4d8293a71479b8c52de62d5a18af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:14.550972 15542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c04dfee972a4f61a4e95ce95051d321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:14.552279 15735 tablet_service.cc:1467] Processing CreateTablet for tablet 2c04dfee972a4f61a4e95ce95051d321 (DEFAULT_TABLE table=test-table [id=6cd4d8293a71479b8c52de62d5a18af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:14.553462 15735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c04dfee972a4f61a4e95ce95051d321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:14.556375 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.534915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46330 (local address 127.25.137.195:38885)
0808 02:47:14.536346 (+ 1431us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.536354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.536387 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:47:14.536486 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.536494 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.536739 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.537884 (+ 1145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:14.538157 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.539795 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.544343 (+ 4548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.545761 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.545946 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.546106 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.546323 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.556037 (+ 9714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1248,"thread_start_us":1104,"threads_started":1}
I20260808 02:47:14.559922 15958 tablet_service.cc:1467] Processing CreateTablet for tablet 2c04dfee972a4f61a4e95ce95051d321 (DEFAULT_TABLE table=test-table [id=6cd4d8293a71479b8c52de62d5a18af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:14.561349 15958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c04dfee972a4f61a4e95ce95051d321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:14.562278 16094 tablet_bootstrap.cc:492] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: Bootstrap starting.
I20260808 02:47:14.565625 16094 tablet_bootstrap.cc:654] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:14.566359 16096 tablet_bootstrap.cc:492] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: Bootstrap starting.
I20260808 02:47:14.567070 16094 log.cc:824] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:14.569636 16096 tablet_bootstrap.cc:654] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:14.570248 16094 tablet_bootstrap.cc:492] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: No bootstrap required, opened a new log
I20260808 02:47:14.570518 16094 ts_tablet_manager.cc:1397] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 02:47:14.570883 16096 log.cc:824] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:14.574223 16100 tablet_bootstrap.cc:492] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: Bootstrap starting.
I20260808 02:47:14.574970 16096 tablet_bootstrap.cc:492] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: No bootstrap required, opened a new log
I20260808 02:47:14.575225 16096 ts_tablet_manager.cc:1397] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260808 02:47:14.577059 16100 tablet_bootstrap.cc:654] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:14.578337 16100 log.cc:824] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:14.580755 16100 tablet_bootstrap.cc:492] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: No bootstrap required, opened a new log
I20260808 02:47:14.581094 16100 ts_tablet_manager.cc:1397] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260808 02:47:14.580891 16094 raft_consensus.cc:348] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:14.581346 16094 raft_consensus.cc:374] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:14.581490 16094 raft_consensus.cc:729] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b1e360420644508f80772cd56ac08b, State: Initialized, Role: FOLLOWER
I20260808 02:47:14.582176 16094 consensus_queue.cc:260] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:14.585443 16096 raft_consensus.cc:348] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:14.585872 16096 raft_consensus.cc:374] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:14.585983 16096 raft_consensus.cc:729] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d9d6454766345c986b8bc519d8432e5, State: Initialized, Role: FOLLOWER
I20260808 02:47:14.586591 16096 consensus_queue.cc:260] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:14.587680 15839 heartbeater.cc:503] Master 127.25.137.254:43729 was elected leader, sending a full tablet report...
I20260808 02:47:14.588385 16094 ts_tablet_manager.cc:1428] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: Time spent starting tablet: real 0.018s user 0.014s sys 0.000s
I20260808 02:47:14.588394 16096 ts_tablet_manager.cc:1428] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: Time spent starting tablet: real 0.013s user 0.006s sys 0.007s
I20260808 02:47:14.590080 16100 raft_consensus.cc:348] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:14.590503 16100 raft_consensus.cc:374] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:14.590595 16100 raft_consensus.cc:729] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c8d38c776aa49099ff618715fa41bb8, State: Initialized, Role: FOLLOWER
I20260808 02:47:14.591125 16100 consensus_queue.cc:260] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:14.592221 16066 heartbeater.cc:503] Master 127.25.137.254:43729 was elected leader, sending a full tablet report...
I20260808 02:47:14.592700 16100 ts_tablet_manager.cc:1428] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
W20260808 02:47:14.653203 15633 tablet.cc:2369] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:47:14.675251 16104 raft_consensus.cc:482] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:47:14.675580 16104 raft_consensus.cc:504] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:14.676695 16103 raft_consensus.cc:482] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:47:14.676954 16103 raft_consensus.cc:504] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:14.677436 16104 leader_election.cc:288] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c8d38c776aa49099ff618715fa41bb8 (127.25.137.195:38885), 49b1e360420644508f80772cd56ac08b (127.25.137.193:33067)
I20260808 02:47:14.678948 16103 leader_election.cc:288] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c8d38c776aa49099ff618715fa41bb8 (127.25.137.195:38885), 8d9d6454766345c986b8bc519d8432e5 (127.25.137.194:36329)
I20260808 02:47:14.687570 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.678039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:57859 (local address 127.25.137.193:33067)
0808 02:47:14.678467 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.678473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.679664 (+ 1191us) server_negotiation.cc:407] Connection header received
0808 02:47:14.679981 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.679987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.680060 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.680118 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:14.680955 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.682158 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.684890 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.686309 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.686442 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.686520 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.687106 (+ 586us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.687266 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270}
I20260808 02:47:14.688351 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.677950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:33067 (local address 127.25.137.194:57859)
0808 02:47:14.679565 (+ 1615us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:14.679582 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:14.679814 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:14.680324 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:14.680328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:14.680353 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:14.680777 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.680788 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.682373 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.682379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.684683 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.684701 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.686721 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.686727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.686914 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.686948 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:47:14.687757 (+ 809us) client_negotiation.cc:240] Negotiation successful
0808 02:47:14.687919 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1490,"mutex_wait_us":1,"thread_start_us":1250,"threads_started":1}
I20260808 02:47:14.688887 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.678627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:59001 (local address 127.25.137.195:38885)
0808 02:47:14.678823 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.678829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.678851 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:47:14.679107 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.679114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.679197 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.679258 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:14.680209 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.681416 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.684329 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.685780 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.685923 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.686001 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.688440 (+ 2439us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.688589 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:47:14.690006 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.677875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:38885 (local address 127.25.137.194:59001)
0808 02:47:14.678697 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:14.678719 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:14.678894 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:14.679560 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:14.679565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:14.679595 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:14.680033 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.680048 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.681792 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.681833 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.684143 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.684158 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.686207 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.686213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.686389 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.686425 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:47:14.689404 (+ 2979us) client_negotiation.cc:240] Negotiation successful
0808 02:47:14.689562 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":3,"spinlock_wait_cycles":64256,"thread_start_us":974,"threads_started":1}
I20260808 02:47:14.690363 15574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c04dfee972a4f61a4e95ce95051d321" candidate_uuid: "8d9d6454766345c986b8bc519d8432e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b1e360420644508f80772cd56ac08b" is_pre_election: true
I20260808 02:47:14.690954 15574 raft_consensus.cc:2454] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d9d6454766345c986b8bc519d8432e5 in term 0.
I20260808 02:47:14.691126 15993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c04dfee972a4f61a4e95ce95051d321" candidate_uuid: "8d9d6454766345c986b8bc519d8432e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8d38c776aa49099ff618715fa41bb8" is_pre_election: true
I20260808 02:47:14.691807 15993 raft_consensus.cc:2454] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d9d6454766345c986b8bc519d8432e5 in term 0.
I20260808 02:47:14.691823 15668 leader_election.cc:302] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49b1e360420644508f80772cd56ac08b, 8d9d6454766345c986b8bc519d8432e5; no voters:
I20260808 02:47:14.692319 16104 raft_consensus.cc:2781] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:47:14.692483 16104 raft_consensus.cc:482] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:47:14.692591 16104 raft_consensus.cc:3037] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:14.695631 16104 raft_consensus.cc:504] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:14.696712 16104 leader_election.cc:288] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [CANDIDATE]: Term 1 election: Requested vote from peers 3c8d38c776aa49099ff618715fa41bb8 (127.25.137.195:38885), 49b1e360420644508f80772cd56ac08b (127.25.137.193:33067)
I20260808 02:47:14.696810 15993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c04dfee972a4f61a4e95ce95051d321" candidate_uuid: "8d9d6454766345c986b8bc519d8432e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8d38c776aa49099ff618715fa41bb8"
I20260808 02:47:14.697074 15993 raft_consensus.cc:3037] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:14.697253 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.679748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:35259 (local address 127.25.137.195:38885)
0808 02:47:14.689098 (+ 9350us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.689104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.689722 (+ 618us) server_negotiation.cc:407] Connection header received
0808 02:47:14.689986 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.689991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.690066 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.690132 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:14.691121 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.692424 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.694853 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.695876 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.696056 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.696151 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.696553 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.696688 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9167,"thread_start_us":8901,"threads_started":1}
I20260808 02:47:14.697671 15574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c04dfee972a4f61a4e95ce95051d321" candidate_uuid: "8d9d6454766345c986b8bc519d8432e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b1e360420644508f80772cd56ac08b"
I20260808 02:47:14.697831 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.679389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:38885 (local address 127.25.137.193:35259)
0808 02:47:14.689614 (+ 10225us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:14.689631 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:14.689832 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:14.690424 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:14.690428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:14.690452 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:14.690884 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.690895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.692579 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.692584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.694672 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.694702 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.696126 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.696133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.696354 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.696390 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:47:14.697081 (+ 691us) client_negotiation.cc:240] Negotiation successful
0808 02:47:14.697272 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10086,"mutex_wait_us":1,"thread_start_us":9897,"threads_started":1}
I20260808 02:47:14.697897 15574 raft_consensus.cc:3037] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:14.698841 15992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c04dfee972a4f61a4e95ce95051d321" candidate_uuid: "49b1e360420644508f80772cd56ac08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8d38c776aa49099ff618715fa41bb8" is_pre_election: true
I20260808 02:47:14.700117 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.684045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:57501 (local address 127.25.137.194:36329)
0808 02:47:14.684236 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.684241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.693137 (+ 8896us) server_negotiation.cc:407] Connection header received
0808 02:47:14.693398 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.693403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.693474 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.693537 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:14.694327 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.695399 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.697777 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.698946 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.699067 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.699132 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.699749 (+ 617us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.699887 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:47:14.700569 15993 raft_consensus.cc:2454] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d9d6454766345c986b8bc519d8432e5 in term 1.
I20260808 02:47:14.700888 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.680229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:36329 (local address 127.25.137.193:57501)
0808 02:47:14.693043 (+ 12814us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:14.693061 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:14.693241 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:14.693727 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:14.693731 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:14.693756 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:14.694131 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.694141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.695538 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.695544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.697576 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:14.697593 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.699362 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.699368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:14.699544 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.699583 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 02:47:14.700343 (+ 760us) client_negotiation.cc:240] Negotiation successful
0808 02:47:14.700488 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12640,"thread_start_us":12461,"threads_started":1}
I20260808 02:47:14.700959 15667 leader_election.cc:302] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c8d38c776aa49099ff618715fa41bb8, 8d9d6454766345c986b8bc519d8432e5; no voters:
I20260808 02:47:14.701092 15574 raft_consensus.cc:2454] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d9d6454766345c986b8bc519d8432e5 in term 1.
I20260808 02:47:14.701327 16104 raft_consensus.cc:2781] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 1 FOLLOWER]: Leader election won for term 1
W20260808 02:47:14.701643 15842 tablet.cc:2369] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:47:14.701984 16104 raft_consensus.cc:686] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 1 LEADER]: Becoming Leader. State: Replica: 8d9d6454766345c986b8bc519d8432e5, State: Running, Role: LEADER
I20260808 02:47:14.702105 15766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c04dfee972a4f61a4e95ce95051d321" candidate_uuid: "49b1e360420644508f80772cd56ac08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d9d6454766345c986b8bc519d8432e5" is_pre_election: true
I20260808 02:47:14.702567 16104 consensus_queue.cc:237] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:14.703106 15462 leader_election.cc:302] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49b1e360420644508f80772cd56ac08b; no voters: 3c8d38c776aa49099ff618715fa41bb8, 8d9d6454766345c986b8bc519d8432e5
I20260808 02:47:14.703519 16103 raft_consensus.cc:2726] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 02:47:14.707646 15341 catalog_manager.cc:5495] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d9d6454766345c986b8bc519d8432e5 (127.25.137.194). New cstate: current_term: 1 leader_uuid: "8d9d6454766345c986b8bc519d8432e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } health_report { overall_health: HEALTHY } } }
W20260808 02:47:14.719491 16067 tablet.cc:2369] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:47:14.736729 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.731699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.25.137.194:36329)
0808 02:47:14.731859 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.731864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.731881 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:47:14.732062 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.732066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.732145 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.732269 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:47:14.732586 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.733390 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.734721 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.735149 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.735241 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.735669 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:47:14.736156 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:47:14.736267 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.736428 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.736495 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:47:14.744959 15574 raft_consensus.cc:1261] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 1 FOLLOWER]: Refusing update from remote peer 8d9d6454766345c986b8bc519d8432e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:47:14.744946 15993 raft_consensus.cc:1261] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8 [term 1 FOLLOWER]: Refusing update from remote peer 8d9d6454766345c986b8bc519d8432e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:47:14.745944 16114 consensus_queue.cc:1035] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:47:14.746450 16104 consensus_queue.cc:1035] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:47:14.758816 16117 mvcc.cc:205] Tried to move back new op lower bound from 7316100033505550336 to 7316100033346134016. Current Snapshot: MvccSnapshot[applied={T|T < 7316100033505550336}]
I20260808 02:47:14.771741 15341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:47:14.772017 15341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:47:14.775779 15341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Sending DeleteTablet for 3 replicas of tablet 2c04dfee972a4f61a4e95ce95051d321
I20260808 02:47:14.777099 15958 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c04dfee972a4f61a4e95ce95051d321 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:14 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:46330
I20260808 02:47:14.777194 15542 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c04dfee972a4f61a4e95ce95051d321 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:14 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:54996
I20260808 02:47:14.777374 15735 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c04dfee972a4f61a4e95ce95051d321 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:14 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:43198
I20260808 02:47:14.778862 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 15434
I20260808 02:47:14.790058 16133 tablet_replica.cc:330] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: stopping tablet replica
I20260808 02:47:14.790634 16133 raft_consensus.cc:2229] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:47:14.791215 16133 raft_consensus.cc:2258] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:14.792994 16130 tablet_replica.cc:330] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: stopping tablet replica
I20260808 02:47:14.793303 16133 ts_tablet_manager.cc:1905] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:14.793442 16130 raft_consensus.cc:2229] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:47:14.793937 16130 raft_consensus.cc:2258] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:14.797792 16130 ts_tablet_manager.cc:1905] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:14.798331 16134 tablet_replica.cc:330] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: stopping tablet replica
I20260808 02:47:14.798735 16134 raft_consensus.cc:2229] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:47:14.799348 16134 raft_consensus.cc:2258] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:14.800722 16133 ts_tablet_manager.cc:1918] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:14.800935 16133 log.cc:1196] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/2c04dfee972a4f61a4e95ce95051d321
I20260808 02:47:14.801534 16133 ts_tablet_manager.cc:1939] T 2c04dfee972a4f61a4e95ce95051d321 P 49b1e360420644508f80772cd56ac08b: Deleting consensus metadata
I20260808 02:47:14.803305 15322 catalog_manager.cc:4841] TS 49b1e360420644508f80772cd56ac08b (127.25.137.193:33067): tablet 2c04dfee972a4f61a4e95ce95051d321 (table test-table [id=6cd4d8293a71479b8c52de62d5a18af1]) successfully deleted
I20260808 02:47:14.804800 16130 ts_tablet_manager.cc:1918] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:14.804958 16130 log.cc:1196] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/2c04dfee972a4f61a4e95ce95051d321
I20260808 02:47:14.805562 16130 ts_tablet_manager.cc:1939] T 2c04dfee972a4f61a4e95ce95051d321 P 3c8d38c776aa49099ff618715fa41bb8: Deleting consensus metadata
I20260808 02:47:14.806365 16134 ts_tablet_manager.cc:1905] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:14.807233 15321 catalog_manager.cc:4841] TS 3c8d38c776aa49099ff618715fa41bb8 (127.25.137.195:38885): tablet 2c04dfee972a4f61a4e95ce95051d321 (table test-table [id=6cd4d8293a71479b8c52de62d5a18af1]) successfully deleted
Aug 08 02:47:14 dist-test-slave-k5rr krb5kdc[15274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:47:14.813546 16134 ts_tablet_manager.cc:1918] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:14.813736 16134 log.cc:1196] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/2c04dfee972a4f61a4e95ce95051d321
I20260808 02:47:14.814288 16134 ts_tablet_manager.cc:1939] T 2c04dfee972a4f61a4e95ce95051d321 P 8d9d6454766345c986b8bc519d8432e5: Deleting consensus metadata
I20260808 02:47:14.816079 15320 catalog_manager.cc:4841] TS 8d9d6454766345c986b8bc519d8432e5 (127.25.137.194:36329): tablet 2c04dfee972a4f61a4e95ce95051d321 (table test-table [id=6cd4d8293a71479b8c52de62d5a18af1]) successfully deleted
I20260808 02:47:14.817586 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.780267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55012 (local address 127.25.137.193:33067)
0808 02:47:14.780451 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.780458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.797023 (+ 16565us) server_negotiation.cc:407] Connection header received
0808 02:47:14.797330 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.797336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.797437 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.797515 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:14.801391 (+ 3876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.802540 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.804599 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.805412 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.805523 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.810989 (+ 5466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:14.811000 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:14.811013 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:14.811046 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:14.813844 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:14.814531 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:14.814538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:14.814540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:14.814662 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:14.815044 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:14.815047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:14.815049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:14.815413 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:14.815691 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.817038 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.817276 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:47:14.818199 15623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:14.909789 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 15434
I20260808 02:47:14.935855 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 15640
Aug 08 02:47:14 dist-test-slave-k5rr krb5kdc[15274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:47:14.958329 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:14.936603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43218 (local address 127.25.137.194:36329)
0808 02:47:14.936924 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:14.936930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:14.936951 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:47:14.937171 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:14.937176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:14.937256 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:14.937347 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:14.938649 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.939399 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.940737 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:14.941239 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:14.941358 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:14.950670 (+ 9312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:14.950685 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:14.950700 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:14.950770 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:14.955010 (+ 4240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:14.955866 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:14.955899 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:14.955903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:14.956095 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:14.956531 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:14.956536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:14.956540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:14.957098 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:14.957401 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:14.957737 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 02:47:14.957970 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:47:14.959064 15831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:15.074885 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 15640
I20260808 02:47:15.100423 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 15850
Aug 08 02:47:15 dist-test-slave-k5rr krb5kdc[15274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:47:15.115547 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:15.101219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.25.137.195:38885)
0808 02:47:15.101415 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:15.101420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:15.101439 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:47:15.101574 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:15.101578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:15.101643 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:15.101724 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:15.102925 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:15.103611 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:15.104887 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:15.105416 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:15.105517 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:15.110008 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:15.110031 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:15.110052 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:15.110088 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:15.112938 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:15.113739 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:15.113746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:15.113749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:15.113890 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:15.114188 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:15.114191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:15.114193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:15.114549 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:15.114774 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:15.115025 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 02:47:15.115216 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 02:47:15.116212 16058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:15.205879 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 15850
I20260808 02:47:15.224890 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 15295
I20260808 02:47:15.226046 15384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:15.336548 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 15295
I20260808 02:47:15.361704 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:43729
--webserver_interface=127.25.137.254
--webserver_port=43819
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:43729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:15.696547 16142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:15.696879 16142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:15.697060 16142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:15.704597 16142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:47:15.704727 16142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:15.704828 16142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:15.704912 16142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:47:15.704970 16142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:47:15.715373 16142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:43729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:43729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=43819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:15.716809 16142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:15.718221 16142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:15.743678 16150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:15.743736 16157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:15.743696 16151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:15.744530 16142 server_base.cc:1034] running on GCE node
I20260808 02:47:15.745409 16142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:15.747006 16142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:15.748245 16142 hybrid_clock.cc:648] HybridClock initialized: now 1786157235748178 us; error 45 us; skew 500 ppm
I20260808 02:47:15.748657 16142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:15 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157235, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:15.755177 16142 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:47:15.756978 16142 webserver.cc:460] Webserver started at http://127.25.137.254:43819/ using document root <none> and password file <none>
I20260808 02:47:15.757761 16142 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:15.757894 16142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:15.763698 16142 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:47:15.766631 16168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:15.768028 16142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 02:47:15.768174 16142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "fb3fb0cb9614444ca57c3d0959d15b50"
format_stamp: "Formatted at 2026-08-08 02:47:12 on dist-test-slave-k5rr"
server_key: "1c64a7e11a492ec312b743bd521df961"
server_key_iv: "d7a7138aeff53c6eddb52d012068bfc9"
server_key_version: "encryptionkey@0"
I20260808 02:47:15.769279 16142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:15.782903 16142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:15.786892 16142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:15.787120 16142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:15.805344 16142 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:43729
I20260808 02:47:15.805411 16255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:43729 every 8 connection(s)
I20260808 02:47:15.806900 16142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:47:15.808549 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 16142
I20260808 02:47:15.809680 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:33067
--local_ip_for_outbound_sockets=127.25.137.193
--tserver_master_addrs=127.25.137.254:43729
--webserver_port=40805
--webserver_interface=127.25.137.193
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:47:15.816635 16257 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:47:15.821349 16257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Bootstrap starting.
I20260808 02:47:15.842162 16257 log.cc:824] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:15.854615 16257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:47:15.855121 16257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Bootstrap complete.
I20260808 02:47:15.864295 16257 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:15.865152 16257 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb3fb0cb9614444ca57c3d0959d15b50, State: Initialized, Role: FOLLOWER
I20260808 02:47:15.865783 16257 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:15.866003 16257 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:47:15.866124 16257 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:47:15.866302 16257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:47:15.870165 16257 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:15.870680 16257 leader_election.cc:302] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb3fb0cb9614444ca57c3d0959d15b50; no voters:
I20260808 02:47:15.871331 16257 leader_election.cc:288] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:47:15.871464 16263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:47:15.872756 16263 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 2 LEADER]: Becoming Leader. State: Replica: fb3fb0cb9614444ca57c3d0959d15b50, State: Running, Role: LEADER
I20260808 02:47:15.873373 16263 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:15.873754 16257 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:47:15.875752 16270 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb3fb0cb9614444ca57c3d0959d15b50. Latest consensus state: current_term: 2 leader_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } } }
I20260808 02:47:15.876446 16270 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:15.876142 16269 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } } }
I20260808 02:47:15.876935 16269 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:15.880286 16275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:47:15.889263 16275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:47:15.890770 16275 catalog_manager.cc:1259] Loaded cluster ID: de8e7cb8501e45d0a13dcb0f592c41f1
I20260808 02:47:15.890872 16275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:47:15.894340 16275 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 02:47:15.914628 16275 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 02:47:15.925122 16275 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:47:15.927054 16275 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Loaded TSK: 0
I20260808 02:47:15.928993 16275 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:47:16.177150 16259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:16.177471 16259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:16.177620 16259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:16.185026 16259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:16.185143 16259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:16.185288 16259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:47:16.195273 16259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:33067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=40805
--tserver_master_addrs=127.25.137.254:43729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:16.196614 16259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:16.198096 16259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:16.212920 16309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:16.213160 16310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:16.213495 16312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:16.214488 16259 server_base.cc:1034] running on GCE node
I20260808 02:47:16.215267 16259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:16.216454 16259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:16.217700 16259 hybrid_clock.cc:648] HybridClock initialized: now 1786157236217633 us; error 54 us; skew 500 ppm
I20260808 02:47:16.218091 16259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:16 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157236, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:16.222220 16259 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:47:16.224033 16259 webserver.cc:460] Webserver started at http://127.25.137.193:40805/ using document root <none> and password file <none>
I20260808 02:47:16.224924 16259 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:16.225099 16259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:16.231701 16259 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:47:16.234962 16324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:16.236445 16259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:47:16.236601 16259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "49b1e360420644508f80772cd56ac08b"
format_stamp: "Formatted at 2026-08-08 02:47:13 on dist-test-slave-k5rr"
server_key: "010819d033aa166e539a440acbceb1b3"
server_key_iv: "e1c6078efe5fd2d823e49194fb7531ff"
server_key_version: "encryptionkey@0"
I20260808 02:47:16.237962 16259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:16.284361 16259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:16.288537 16259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:16.288771 16259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:16.289784 16259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:16.291132 16259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:16.291229 16259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:16.291311 16259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:16.291364 16259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:16.322285 16259 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:33067
I20260808 02:47:16.322409 16508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:33067 every 8 connection(s)
I20260808 02:47:16.323772 16259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:47:16.330402 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 16259
I20260808 02:47:16.331172 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:36329
--local_ip_for_outbound_sockets=127.25.137.194
--tserver_master_addrs=127.25.137.254:43729
--webserver_port=36553
--webserver_interface=127.25.137.194
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:47:16 dist-test-slave-k5rr krb5kdc[15274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157236, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:16.348126 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:16.327165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:54979)
0808 02:47:16.328097 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:16.328156 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:16.330189 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:16.331759 (+ 1570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:16.331770 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:16.332365 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:16.332886 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:16.332903 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:16.334513 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:16.334517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:16.335713 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:16.335726 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:16.336714 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:16.336722 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:16.336971 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:16.337661 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:16.337684 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:16.340057 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:16.343963 (+ 3906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:16.343972 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:16.343987 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:16.344624 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:16.345097 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:16.345100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:16.345102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:16.345281 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:16.346177 (+ 896us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:16.346184 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:16.346472 (+ 288us) client_negotiation.cc:769] Sending connection context
0808 02:47:16.346702 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 02:47:16.347116 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":349,"threads_started":1}
I20260808 02:47:16.348439 16513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:16.326917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:54979 (local address 127.25.137.254:43729)
0808 02:47:16.327504 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:16.327514 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:16.328275 (+ 761us) server_negotiation.cc:407] Connection header received
0808 02:47:16.330834 (+ 2559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:16.330848 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:16.331186 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:16.331576 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:16.333129 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:16.334361 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:16.335854 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:16.336480 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:16.336663 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:16.340339 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:16.340347 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:16.340360 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:16.340397 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:16.343791 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:16.344764 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:16.344780 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:16.344788 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:16.344977 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:16.345394 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:16.345399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:16.345401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:16.346010 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:16.346185 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:16.346955 (+ 770us) server_negotiation.cc:299] Negotiation successful
0808 02:47:16.347375 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":256,"threads_started":1}
I20260808 02:47:16.351212 16509 heartbeater.cc:347] Connected to a master server at 127.25.137.254:43729
I20260808 02:47:16.351508 16509 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:16.352221 16509 heartbeater.cc:511] Master 127.25.137.254:43729 requested a full tablet report, sending...
I20260808 02:47:16.354672 16195 ts_manager.cc:194] Registered new tserver with Master: 49b1e360420644508f80772cd56ac08b (127.25.137.193:33067)
I20260808 02:47:16.356709 16195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:54979
W20260808 02:47:16.670456 16516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:16.670755 16516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:16.670881 16516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:16.678751 16516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:16.678874 16516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:16.679006 16516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:47:16.689293 16516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:36329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=36553
--tserver_master_addrs=127.25.137.254:43729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:16.690641 16516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:16.692034 16516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:16.705828 16523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:16.705896 16525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:16.705865 16522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:16.706739 16516 server_base.cc:1034] running on GCE node
I20260808 02:47:16.707433 16516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:16.708417 16516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:16.709672 16516 hybrid_clock.cc:648] HybridClock initialized: now 1786157236709610 us; error 42 us; skew 500 ppm
I20260808 02:47:16.710006 16516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:16 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157236, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:16.713331 16516 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:47:16.714792 16516 webserver.cc:460] Webserver started at http://127.25.137.194:36553/ using document root <none> and password file <none>
I20260808 02:47:16.715487 16516 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:16.715637 16516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:16.721271 16516 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:47:16.723996 16536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:16.725471 16516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:47:16.725623 16516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "8d9d6454766345c986b8bc519d8432e5"
format_stamp: "Formatted at 2026-08-08 02:47:13 on dist-test-slave-k5rr"
server_key: "1aca603eef7e19a51969830fe22bc225"
server_key_iv: "bbb64766eb54a39d14220df977fb61cb"
server_key_version: "encryptionkey@0"
I20260808 02:47:16.726668 16516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:16.762887 16516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:16.767305 16516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:16.767563 16516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:16.768571 16516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:16.769948 16516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:16.770045 16516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:16.770156 16516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:16.770227 16516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:16.800405 16516 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:36329
I20260808 02:47:16.800472 16707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:36329 every 8 connection(s)
I20260808 02:47:16.801994 16516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:47:16.812155 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 16516
I20260808 02:47:16.812887 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:38885
--local_ip_for_outbound_sockets=127.25.137.195
--tserver_master_addrs=127.25.137.254:43729
--webserver_port=39319
--webserver_interface=127.25.137.195
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:47:16 dist-test-slave-k5rr krb5kdc[15274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157236, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:16.822403 16513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:16.804539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:42495 (local address 127.25.137.254:43729)
0808 02:47:16.804777 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:16.804781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:16.806182 (+ 1401us) server_negotiation.cc:407] Connection header received
0808 02:47:16.808256 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:16.808263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:16.808354 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:16.808470 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:16.810056 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:16.811216 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:16.812860 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:16.813469 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:16.813588 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:16.816863 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:16.816868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:16.816872 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:16.816905 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:16.819465 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:16.820151 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:16.820155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:16.820158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:16.820283 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:16.820691 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:16.820695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:16.820697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:16.820984 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:16.821146 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:16.821823 (+ 677us) server_negotiation.cc:299] Negotiation successful
0808 02:47:16.822058 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:47:16.823366 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:16.805183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:42495)
0808 02:47:16.806014 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:16.806061 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:16.807970 (+ 1909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:16.808705 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:16.808713 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:16.809332 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:16.809886 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:16.809904 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:16.811388 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:16.811392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:16.812700 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:16.812713 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:16.813716 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:16.813725 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:16.813943 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:16.814554 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:16.814581 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:16.816653 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:16.819607 (+ 2954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:16.819616 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:16.819630 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:16.820026 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:16.820393 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:16.820398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:16.820402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:16.820584 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:16.821128 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:16.821135 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:16.821456 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 02:47:16.821694 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 02:47:16.822190 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":319,"threads_started":1}
I20260808 02:47:16.824785 16708 heartbeater.cc:347] Connected to a master server at 127.25.137.254:43729
I20260808 02:47:16.825137 16708 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:16.825862 16708 heartbeater.cc:511] Master 127.25.137.254:43729 requested a full tablet report, sending...
I20260808 02:47:16.827286 16195 ts_manager.cc:194] Registered new tserver with Master: 8d9d6454766345c986b8bc519d8432e5 (127.25.137.194:36329)
I20260808 02:47:16.828065 16195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:42495
W20260808 02:47:17.145824 16711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:17.146121 16711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:17.146260 16711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:17.153503 16711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:17.153630 16711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:17.153796 16711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:47:17.163481 16711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:38885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=39319
--tserver_master_addrs=127.25.137.254:43729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:17.164865 16711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:17.166271 16711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:17.179386 16722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:17.179318 16723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:17.179354 16727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:17.180059 16711 server_base.cc:1034] running on GCE node
I20260808 02:47:17.180630 16711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:17.181567 16711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:17.182760 16711 hybrid_clock.cc:648] HybridClock initialized: now 1786157237182699 us; error 51 us; skew 500 ppm
I20260808 02:47:17.183072 16711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:17 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157237, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:17.186126 16711 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:47:17.187578 16711 webserver.cc:460] Webserver started at http://127.25.137.195:39319/ using document root <none> and password file <none>
I20260808 02:47:17.188235 16711 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:17.188380 16711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:17.193812 16711 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:47:17.196458 16737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:17.197798 16711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:47:17.197957 16711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "3c8d38c776aa49099ff618715fa41bb8"
format_stamp: "Formatted at 2026-08-08 02:47:14 on dist-test-slave-k5rr"
server_key: "c2cb30737da16a77456262b3d7ec85c1"
server_key_iv: "da2e25ec69ebb5c21b8e38a740899c2d"
server_key_version: "encryptionkey@0"
I20260808 02:47:17.198915 16711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:17.226732 16711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:17.230741 16711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:17.231045 16711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:17.232043 16711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:17.233469 16711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:17.233574 16711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:17.233685 16711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:17.233760 16711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:17.264487 16711 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:38885
I20260808 02:47:17.264577 16935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:38885 every 8 connection(s)
I20260808 02:47:17.265972 16711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:47:17.273375 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 16711
Aug 08 02:47:17 dist-test-slave-k5rr krb5kdc[15274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157237, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:17.288807 16941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.274252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52408 (local address 127.25.137.254:43729)
0808 02:47:17.274735 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.274740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.275154 (+ 414us) server_negotiation.cc:407] Connection header received
0808 02:47:17.275317 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.275322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.275394 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.275511 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:17.276962 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.277786 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.279337 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.279991 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.280121 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.282954 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:17.282962 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:17.282967 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:17.283021 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:17.285962 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:17.286698 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:17.286702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:17.286705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:17.286859 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:17.287266 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:17.287271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:17.287274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:17.287521 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:17.287703 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.288101 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.288344 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":371,"spinlock_wait_cycles":1920,"thread_start_us":251,"threads_started":1}
I20260808 02:47:17.288807 16513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.268946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:42387 (local address 127.25.137.254:43729)
0808 02:47:17.269157 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.269162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.270271 (+ 1109us) server_negotiation.cc:407] Connection header received
0808 02:47:17.272415 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.272422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.272510 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.272600 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:17.274391 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.275534 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.277254 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.277779 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.277915 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.281759 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:17.281764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:17.281767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:17.281807 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:17.285128 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:17.285941 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:17.285946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:17.285950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:17.286085 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:17.286672 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:17.286678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:17.286681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:17.287135 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:17.287261 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.288094 (+ 833us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.288469 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":183,"server-negotiator.queue_time_us":90}
I20260808 02:47:17.289937 16940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.269401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:42387)
0808 02:47:17.270110 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:17.270159 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:17.272089 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:17.272825 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:17.272833 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:17.273450 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:17.274221 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.274242 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.275719 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.275723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.277040 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.277056 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.278014 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.278020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.278227 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.278874 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:17.278896 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:17.281545 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:17.285308 (+ 3763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:17.285316 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:17.285329 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:17.285799 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:17.286215 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:17.286220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:17.286224 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:17.286506 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:17.287266 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:17.287274 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:17.287641 (+ 367us) client_negotiation.cc:769] Sending connection context
0808 02:47:17.287970 (+ 329us) client_negotiation.cc:240] Negotiation successful
0808 02:47:17.288575 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":260,"threads_started":1}
I20260808 02:47:17.291445 16936 heartbeater.cc:347] Connected to a master server at 127.25.137.254:43729
I20260808 02:47:17.291909 16936 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:17.292618 16936 heartbeater.cc:511] Master 127.25.137.254:43729 requested a full tablet report, sending...
I20260808 02:47:17.293995 16195 ts_manager.cc:194] Registered new tserver with Master: 3c8d38c776aa49099ff618715fa41bb8 (127.25.137.195:38885)
I20260808 02:47:17.294838 16195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:42387
I20260808 02:47:17.303858 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:47:17.310503 16941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.304771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52418 (local address 127.25.137.254:43729)
0808 02:47:17.304948 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.304952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.305416 (+ 464us) server_negotiation.cc:407] Connection header received
0808 02:47:17.305588 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.305592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.305763 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.305904 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:47:17.306276 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.307034 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.308477 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.308883 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.308971 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.309491 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:47:17.309987 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:47:17.310105 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.310208 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.310274 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:47:17.311569 16195 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:47:17.318830 16195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:47:17.352662 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.331703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:36329 (local address 127.0.0.1:43220)
0808 02:47:17.344871 (+ 13168us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:17.344886 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:17.345271 (+ 385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:17.345863 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:17.345867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:17.345892 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:17.346215 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.346223 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.347482 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.347488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.349950 (+ 2462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.349972 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.351587 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.351594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.351781 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.351841 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 02:47:17.351980 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 02:47:17.352169 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13065,"mutex_wait_us":141,"thread_start_us":10890,"threads_started":1}
I20260808 02:47:17.353137 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.331073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.25.137.193:33067)
0808 02:47:17.343282 (+ 12209us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.343292 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.343326 (+ 34us) server_negotiation.cc:407] Connection header received
0808 02:47:17.343434 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.343442 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.343723 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.343944 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:17.345764 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.347394 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.350651 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.351663 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.351863 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.352031 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.352602 (+ 571us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.352774 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12044,"thread_start_us":11917,"threads_started":1}
I20260808 02:47:17.353252 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.331288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:38885 (local address 127.0.0.1:46350)
0808 02:47:17.342889 (+ 11601us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:17.342910 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:17.343042 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:17.344447 (+ 1405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:17.344452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:17.344480 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:17.345521 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.345532 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.347549 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.347555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.350451 (+ 2896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.350467 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.352443 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.352449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.352616 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.352650 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:47:17.352726 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:47:17.352842 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11458,"mutex_wait_us":211,"thread_start_us":8857,"threads_started":1}
I20260808 02:47:17.353763 16958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.331011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:33067 (local address 127.0.0.1:55022)
0808 02:47:17.340583 (+ 9572us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:17.340637 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:17.340856 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:17.344248 (+ 3392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:17.344258 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:17.344307 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:17.345492 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.345510 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.347549 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.347555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.350451 (+ 2896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.350467 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.352184 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.352190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.352371 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.352407 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:47:17.353320 (+ 913us) client_negotiation.cc:240] Negotiation successful
0808 02:47:17.353444 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9290,"mutex_wait_us":268,"thread_start_us":13615,"threads_started":1}
I20260808 02:47:17.353995 16963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.331903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46350 (local address 127.25.137.195:38885)
0808 02:47:17.343691 (+ 11788us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.343699 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.343730 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:47:17.343818 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.343826 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.344040 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.344347 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:17.345692 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.347371 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.350618 (+ 3247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.351920 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.352109 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.352268 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.353526 (+ 1258us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.353689 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11612,"thread_start_us":11467,"threads_started":1}
I20260808 02:47:17.354282 16966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.332301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43220 (local address 127.25.137.194:36329)
0808 02:47:17.343647 (+ 11346us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.343653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.344993 (+ 1340us) server_negotiation.cc:407] Connection header received
0808 02:47:17.345469 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.345476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.345690 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.345878 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:17.346365 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.347346 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.350110 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.351357 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.351536 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.351709 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.353567 (+ 1858us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.353716 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9930,"thread_start_us":9789,"threads_started":1}
I20260808 02:47:17.357261 16608 tablet_service.cc:1467] Processing CreateTablet for tablet db63bdf17bd44b04b094286a1d56085a (DEFAULT_TABLE table=test-table [id=054f2fa4206f41978593e9fd813385e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:17.357517 16406 tablet_service.cc:1467] Processing CreateTablet for tablet db63bdf17bd44b04b094286a1d56085a (DEFAULT_TABLE table=test-table [id=054f2fa4206f41978593e9fd813385e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:17.357519 16826 tablet_service.cc:1467] Processing CreateTablet for tablet db63bdf17bd44b04b094286a1d56085a (DEFAULT_TABLE table=test-table [id=054f2fa4206f41978593e9fd813385e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:17.358444 16608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db63bdf17bd44b04b094286a1d56085a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:17.358878 16406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db63bdf17bd44b04b094286a1d56085a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:17.359189 16826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db63bdf17bd44b04b094286a1d56085a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:17.359817 16509 heartbeater.cc:503] Master 127.25.137.254:43729 was elected leader, sending a full tablet report...
I20260808 02:47:17.370833 16968 tablet_bootstrap.cc:492] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: Bootstrap starting.
I20260808 02:47:17.374596 16968 tablet_bootstrap.cc:654] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:17.375779 16968 log.cc:824] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:17.376317 16973 tablet_bootstrap.cc:492] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: Bootstrap starting.
I20260808 02:47:17.377848 16970 tablet_bootstrap.cc:492] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: Bootstrap starting.
I20260808 02:47:17.380378 16973 tablet_bootstrap.cc:654] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:17.380548 16970 tablet_bootstrap.cc:654] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:17.381743 16970 log.cc:824] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:17.382511 16973 log.cc:824] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:17.384642 16970 tablet_bootstrap.cc:492] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: No bootstrap required, opened a new log
I20260808 02:47:17.384876 16970 ts_tablet_manager.cc:1397] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:47:17.385219 16973 tablet_bootstrap.cc:492] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: No bootstrap required, opened a new log
I20260808 02:47:17.385492 16973 ts_tablet_manager.cc:1397] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260808 02:47:17.386725 16968 tablet_bootstrap.cc:492] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: No bootstrap required, opened a new log
I20260808 02:47:17.387001 16968 ts_tablet_manager.cc:1397] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260808 02:47:17.392467 16970 raft_consensus.cc:348] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:17.392939 16970 raft_consensus.cc:374] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:17.393075 16970 raft_consensus.cc:729] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b1e360420644508f80772cd56ac08b, State: Initialized, Role: FOLLOWER
I20260808 02:47:17.393697 16970 consensus_queue.cc:260] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:17.395891 16973 raft_consensus.cc:348] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:17.396293 16973 raft_consensus.cc:374] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:17.396394 16973 raft_consensus.cc:729] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c8d38c776aa49099ff618715fa41bb8, State: Initialized, Role: FOLLOWER
I20260808 02:47:17.397325 16970 ts_tablet_manager.cc:1428] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260808 02:47:17.397809 16968 raft_consensus.cc:348] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:17.398247 16968 raft_consensus.cc:374] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:17.398342 16968 raft_consensus.cc:729] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d9d6454766345c986b8bc519d8432e5, State: Initialized, Role: FOLLOWER
I20260808 02:47:17.398962 16968 consensus_queue.cc:260] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:17.399549 16973 consensus_queue.cc:260] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:17.407182 16968 ts_tablet_manager.cc:1428] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: Time spent starting tablet: real 0.020s user 0.016s sys 0.004s
I20260808 02:47:17.407589 16708 heartbeater.cc:503] Master 127.25.137.254:43729 was elected leader, sending a full tablet report...
I20260808 02:47:17.407812 16936 heartbeater.cc:503] Master 127.25.137.254:43729 was elected leader, sending a full tablet report...
I20260808 02:47:17.408157 16973 ts_tablet_manager.cc:1428] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: Time spent starting tablet: real 0.023s user 0.015s sys 0.004s
W20260808 02:47:17.517838 16938 tablet.cc:2369] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:47:17.553995 16709 tablet.cc:2369] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:47:17.576301 16511 tablet.cc:2369] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:47:17.733827 16982 raft_consensus.cc:482] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:47:17.734157 16982 raft_consensus.cc:504] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:17.735948 16982 leader_election.cc:288] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c8d38c776aa49099ff618715fa41bb8 (127.25.137.195:38885), 8d9d6454766345c986b8bc519d8432e5 (127.25.137.194:36329)
I20260808 02:47:17.736052 16989 raft_consensus.cc:482] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:47:17.736523 16989 raft_consensus.cc:504] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:17.745694 16940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.738808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:33067 (local address 127.25.137.195:55845)
0808 02:47:17.739078 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:17.739095 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:17.739181 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:17.739599 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:17.739603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:17.739627 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:17.739857 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.739868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.741300 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.741305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.743189 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.743205 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.744861 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.744867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.745139 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.745169 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 02:47:17.745252 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 02:47:17.745389 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":1}
I20260808 02:47:17.745831 16966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.736820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:52183 (local address 127.25.137.194:36329)
0808 02:47:17.737000 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.737037 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.737655 (+ 618us) server_negotiation.cc:407] Connection header received
0808 02:47:17.737967 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.737980 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.738065 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.738177 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:17.738996 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.740041 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.743145 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.744494 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.744621 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.744698 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.745384 (+ 686us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.745532 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 02:47:17.746228 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.736371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:38885 (local address 127.25.137.193:48083)
0808 02:47:17.737876 (+ 1505us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:17.737891 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:17.737997 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:17.738386 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:17.738390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:17.738431 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:17.738944 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.738959 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.740415 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.740421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.743401 (+ 2980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.743414 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.745128 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.745134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.745304 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.745335 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:47:17.745726 (+ 391us) client_negotiation.cc:240] Negotiation successful
0808 02:47:17.745865 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1394,"mutex_wait_us":4,"thread_start_us":817,"threads_started":1}
I20260808 02:47:17.746673 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.738944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:55845 (local address 127.25.137.193:33067)
0808 02:47:17.739121 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.739126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.739142 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:47:17.739307 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.739312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.739375 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.739453 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:17.740009 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.741110 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.743419 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.744575 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.744715 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.744784 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.746252 (+ 1468us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.746475 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:47:17.746151 16963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.736615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:48083 (local address 127.25.137.195:38885)
0808 02:47:17.736821 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.736826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.737952 (+ 1126us) server_negotiation.cc:407] Connection header received
0808 02:47:17.738116 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.738120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.738192 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.738284 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:17.739160 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.740206 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.743567 (+ 3361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.744717 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.744844 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.745071 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.745466 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.745590 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":143,"server-negotiator.queue_time_us":75}
I20260808 02:47:17.746448 16989 leader_election.cc:288] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49b1e360420644508f80772cd56ac08b (127.25.137.193:33067), 8d9d6454766345c986b8bc519d8432e5 (127.25.137.194:36329)
I20260808 02:47:17.747650 16438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63bdf17bd44b04b094286a1d56085a" candidate_uuid: "3c8d38c776aa49099ff618715fa41bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b1e360420644508f80772cd56ac08b" is_pre_election: true
I20260808 02:47:17.747650 16853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63bdf17bd44b04b094286a1d56085a" candidate_uuid: "49b1e360420644508f80772cd56ac08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8d38c776aa49099ff618715fa41bb8" is_pre_election: true
I20260808 02:47:17.748095 16853 raft_consensus.cc:2454] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49b1e360420644508f80772cd56ac08b in term 0.
I20260808 02:47:17.748100 16438 raft_consensus.cc:2454] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c8d38c776aa49099ff618715fa41bb8 in term 0.
I20260808 02:47:17.748742 16329 leader_election.cc:302] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c8d38c776aa49099ff618715fa41bb8, 49b1e360420644508f80772cd56ac08b; no voters:
I20260808 02:47:17.748950 16744 leader_election.cc:302] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c8d38c776aa49099ff618715fa41bb8, 49b1e360420644508f80772cd56ac08b; no voters:
I20260808 02:47:17.749292 16982 raft_consensus.cc:2781] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:47:17.749372 16989 raft_consensus.cc:2781] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:47:17.749439 16982 raft_consensus.cc:482] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:47:17.749522 16989 raft_consensus.cc:482] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:47:17.749549 16982 raft_consensus.cc:3037] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:17.749625 16989 raft_consensus.cc:3037] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:17.750047 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.736309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:36329 (local address 127.25.137.193:52183)
0808 02:47:17.737539 (+ 1230us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:17.737562 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:17.737789 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:17.738235 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:17.738240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:17.738265 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:17.738799 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.738819 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.740215 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.740220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.742946 (+ 2726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.742963 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.744949 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.744956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.745188 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.745223 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:47:17.749523 (+ 4300us) client_negotiation.cc:240] Negotiation successful
0808 02:47:17.749659 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"spinlock_wait_cycles":20864,"thread_start_us":1279,"threads_started":1}
I20260808 02:47:17.752779 16989 raft_consensus.cc:504] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:17.752842 16982 raft_consensus.cc:504] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:17.753638 16982 leader_election.cc:288] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [CANDIDATE]: Term 1 election: Requested vote from peers 3c8d38c776aa49099ff618715fa41bb8 (127.25.137.195:38885), 8d9d6454766345c986b8bc519d8432e5 (127.25.137.194:36329)
I20260808 02:47:17.753563 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63bdf17bd44b04b094286a1d56085a" candidate_uuid: "49b1e360420644508f80772cd56ac08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d9d6454766345c986b8bc519d8432e5" is_pre_election: true
I20260808 02:47:17.753918 16853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63bdf17bd44b04b094286a1d56085a" candidate_uuid: "49b1e360420644508f80772cd56ac08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8d38c776aa49099ff618715fa41bb8"
I20260808 02:47:17.754173 16853 raft_consensus.cc:2379] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 49b1e360420644508f80772cd56ac08b in current term 1: Already voted for candidate 3c8d38c776aa49099ff618715fa41bb8 in this term.
I20260808 02:47:17.754168 16636 raft_consensus.cc:2454] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49b1e360420644508f80772cd56ac08b in term 0.
I20260808 02:47:17.754328 16632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63bdf17bd44b04b094286a1d56085a" candidate_uuid: "49b1e360420644508f80772cd56ac08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d9d6454766345c986b8bc519d8432e5"
I20260808 02:47:17.754166 16966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.739958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:36145 (local address 127.25.137.194:36329)
0808 02:47:17.746055 (+ 6097us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.746061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.746078 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:47:17.746134 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.746138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.746208 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.746291 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:17.747074 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.748269 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.750777 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.751999 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.752129 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.752195 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.752736 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.752884 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5998,"thread_start_us":12622,"threads_started":1}
I20260808 02:47:17.754591 16632 raft_consensus.cc:3037] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:17.754726 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.739656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:36329 (local address 127.25.137.195:36145)
0808 02:47:17.741114 (+ 1458us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:17.741130 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:17.741224 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:17.746438 (+ 5214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:17.746443 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:17.746468 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:17.746859 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.746870 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.748477 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.748483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.750591 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:17.750609 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.752369 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.752374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:17.752548 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.752579 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:47:17.754235 (+ 1656us) client_negotiation.cc:240] Negotiation successful
0808 02:47:17.754369 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1336,"thread_start_us":1221,"threads_started":1}
I20260808 02:47:17.754930 16989 leader_election.cc:288] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [CANDIDATE]: Term 1 election: Requested vote from peers 49b1e360420644508f80772cd56ac08b (127.25.137.193:33067), 8d9d6454766345c986b8bc519d8432e5 (127.25.137.194:36329)
I20260808 02:47:17.753942 16438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63bdf17bd44b04b094286a1d56085a" candidate_uuid: "3c8d38c776aa49099ff618715fa41bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b1e360420644508f80772cd56ac08b"
I20260808 02:47:17.755288 16438 raft_consensus.cc:2379] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c8d38c776aa49099ff618715fa41bb8 in current term 1: Already voted for candidate 49b1e360420644508f80772cd56ac08b in this term.
I20260808 02:47:17.755919 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63bdf17bd44b04b094286a1d56085a" candidate_uuid: "3c8d38c776aa49099ff618715fa41bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d9d6454766345c986b8bc519d8432e5" is_pre_election: true
I20260808 02:47:17.756099 16631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63bdf17bd44b04b094286a1d56085a" candidate_uuid: "3c8d38c776aa49099ff618715fa41bb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d9d6454766345c986b8bc519d8432e5"
I20260808 02:47:17.756878 16742 leader_election.cc:302] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c8d38c776aa49099ff618715fa41bb8; no voters: 49b1e360420644508f80772cd56ac08b, 8d9d6454766345c986b8bc519d8432e5
I20260808 02:47:17.757259 16989 raft_consensus.cc:2726] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 02:47:17.757624 16632 raft_consensus.cc:2454] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49b1e360420644508f80772cd56ac08b in term 1.
I20260808 02:47:17.757953 16328 leader_election.cc:302] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49b1e360420644508f80772cd56ac08b, 8d9d6454766345c986b8bc519d8432e5; no voters: 3c8d38c776aa49099ff618715fa41bb8
I20260808 02:47:17.758275 16982 raft_consensus.cc:2781] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:47:17.758751 16982 raft_consensus.cc:686] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 1 LEADER]: Becoming Leader. State: Replica: 49b1e360420644508f80772cd56ac08b, State: Running, Role: LEADER
I20260808 02:47:17.759215 16982 consensus_queue.cc:237] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } }
I20260808 02:47:17.763520 16195 catalog_manager.cc:5495] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b reported cstate change: term changed from 0 to 1, leader changed from <none> to 49b1e360420644508f80772cd56ac08b (127.25.137.193). New cstate: current_term: 1 leader_uuid: "49b1e360420644508f80772cd56ac08b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b1e360420644508f80772cd56ac08b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 33067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 } health_report { overall_health: UNKNOWN } } }
I20260808 02:47:17.811550 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.804448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55026 (local address 127.25.137.193:33067)
0808 02:47:17.804657 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.804664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.804685 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:47:17.804813 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.804819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.804915 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.805081 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:47:17.805581 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.806721 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.808780 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.809461 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.809593 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.810165 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:47:17.810816 (+ 651us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:47:17.810950 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.811153 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.811242 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:47:17.820240 16632 raft_consensus.cc:1261] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5 [term 1 FOLLOWER]: Refusing update from remote peer 49b1e360420644508f80772cd56ac08b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:47:17.820283 16853 raft_consensus.cc:1261] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 1 FOLLOWER]: Refusing update from remote peer 49b1e360420644508f80772cd56ac08b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:47:17.821164 17000 consensus_queue.cc:1035] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d9d6454766345c986b8bc519d8432e5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:47:17.821496 16982 consensus_queue.cc:1035] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c8d38c776aa49099ff618715fa41bb8" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:47:17.848065 16195 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:47:17.848381 16195 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:47:17.852146 16195 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Sending DeleteTablet for 3 replicas of tablet db63bdf17bd44b04b094286a1d56085a
I20260808 02:47:17.853317 16406 tablet_service.cc:1514] Processing DeleteTablet for tablet db63bdf17bd44b04b094286a1d56085a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:17 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:55022
I20260808 02:47:17.853453 16826 tablet_service.cc:1514] Processing DeleteTablet for tablet db63bdf17bd44b04b094286a1d56085a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:17 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:46350
I20260808 02:47:17.854084 16608 tablet_service.cc:1514] Processing DeleteTablet for tablet db63bdf17bd44b04b094286a1d56085a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:17 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:43220
I20260808 02:47:17.855103 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 16259
I20260808 02:47:17.866743 17025 tablet_replica.cc:330] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: stopping tablet replica
I20260808 02:47:17.867017 17023 tablet_replica.cc:330] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: stopping tablet replica
I20260808 02:47:17.867383 17025 raft_consensus.cc:2229] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:47:17.867580 17023 raft_consensus.cc:2229] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:47:17.868000 17025 raft_consensus.cc:2258] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:17.868119 17023 raft_consensus.cc:2258] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:17.868834 17021 tablet_replica.cc:330] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: stopping tablet replica
I20260808 02:47:17.869299 17021 raft_consensus.cc:2229] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:47:17.870079 17021 raft_consensus.cc:2258] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:17.870844 17023 ts_tablet_manager.cc:1905] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:17.872212 17025 ts_tablet_manager.cc:1905] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:17.873332 17021 ts_tablet_manager.cc:1905] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:17.877884 17023 ts_tablet_manager.cc:1918] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:17.878100 17023 log.cc:1196] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/db63bdf17bd44b04b094286a1d56085a
I20260808 02:47:17.878703 17023 ts_tablet_manager.cc:1939] T db63bdf17bd44b04b094286a1d56085a P 3c8d38c776aa49099ff618715fa41bb8: Deleting consensus metadata
I20260808 02:47:17.879302 17025 ts_tablet_manager.cc:1918] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:17.879527 17025 log.cc:1196] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/db63bdf17bd44b04b094286a1d56085a
I20260808 02:47:17.880125 17025 ts_tablet_manager.cc:1939] T db63bdf17bd44b04b094286a1d56085a P 8d9d6454766345c986b8bc519d8432e5: Deleting consensus metadata
I20260808 02:47:17.880671 16174 catalog_manager.cc:4841] TS 3c8d38c776aa49099ff618715fa41bb8 (127.25.137.195:38885): tablet db63bdf17bd44b04b094286a1d56085a (table test-table [id=054f2fa4206f41978593e9fd813385e6]) successfully deleted
I20260808 02:47:17.881845 16173 catalog_manager.cc:4841] TS 8d9d6454766345c986b8bc519d8432e5 (127.25.137.194:36329): tablet db63bdf17bd44b04b094286a1d56085a (table test-table [id=054f2fa4206f41978593e9fd813385e6]) successfully deleted
I20260808 02:47:17.882710 17021 ts_tablet_manager.cc:1918] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:17.882885 17021 log.cc:1196] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/db63bdf17bd44b04b094286a1d56085a
I20260808 02:47:17.883495 17021 ts_tablet_manager.cc:1939] T db63bdf17bd44b04b094286a1d56085a P 49b1e360420644508f80772cd56ac08b: Deleting consensus metadata
I20260808 02:47:17.885195 16175 catalog_manager.cc:4841] TS 49b1e360420644508f80772cd56ac08b (127.25.137.193:33067): tablet db63bdf17bd44b04b094286a1d56085a (table test-table [id=054f2fa4206f41978593e9fd813385e6]) successfully deleted
I20260808 02:47:17.890321 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:17.856583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55036 (local address 127.25.137.193:33067)
0808 02:47:17.856789 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:17.856796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:17.862524 (+ 5728us) server_negotiation.cc:407] Connection header received
0808 02:47:17.863477 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:17.863483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:17.863575 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:17.863669 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:17.868514 (+ 4845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.869678 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.873561 (+ 3883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:17.874219 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:17.874352 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:17.883951 (+ 9599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:17.883967 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:17.883980 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:17.884016 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:17.887110 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:17.887933 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:17.887943 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:17.887947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:17.888119 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:17.888551 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:17.888557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:17.888560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:17.889144 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:17.889486 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:17.889734 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 02:47:17.889979 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:47:17.891039 16504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:17.994791 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 16259
I20260808 02:47:18.022272 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 16516
I20260808 02:47:18.036049 16966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:18.023083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43228 (local address 127.25.137.194:36329)
0808 02:47:18.023296 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:18.023300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:18.023322 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:47:18.023486 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:18.023491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:18.023589 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:18.023680 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:18.024987 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:18.025769 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:18.027140 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:18.027563 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:18.027668 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:18.030265 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:18.030281 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:18.030298 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:18.030335 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:18.033479 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:18.034150 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:18.034158 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:18.034160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:18.034327 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:18.034626 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:18.034629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:18.034631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:18.035021 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:18.035276 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:18.035523 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 02:47:18.035696 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:47:18.036716 16700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:18.128150 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 16516
I20260808 02:47:18.152112 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 16711
I20260808 02:47:18.166184 16963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:18.153109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46362 (local address 127.25.137.195:38885)
0808 02:47:18.153321 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:18.153328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:18.153350 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:47:18.153497 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:18.153502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:18.153581 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:18.153667 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:18.155065 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:18.155808 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:18.157255 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:18.157670 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:18.157781 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:18.160418 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:18.160435 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:18.160451 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:18.160486 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:18.163265 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:18.163969 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:18.163977 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:18.163980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:18.164119 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:18.164520 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:18.164524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:18.164526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:18.164920 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:18.165224 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:18.165595 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 02:47:18.165813 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:47:18.166831 16923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:18.256443 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 16711
I20260808 02:47:18.281122 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 16142
I20260808 02:47:18.282298 16244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:18.395370 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 16142
I20260808 02:47:18.424688 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:43729
--webserver_interface=127.25.137.254
--webserver_port=43819
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:43729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:18.754325 17034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:18.754624 17034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:18.754762 17034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:18.761863 17034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:47:18.761977 17034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:18.762043 17034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:18.762123 17034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:47:18.762179 17034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:47:18.771869 17034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:43729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:43729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=43819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:18.773172 17034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:18.774402 17034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:18.785352 17046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:18.785422 17042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:18.785738 17041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:18.786500 17034 server_base.cc:1034] running on GCE node
I20260808 02:47:18.787241 17034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:18.788591 17034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:18.789788 17034 hybrid_clock.cc:648] HybridClock initialized: now 1786157238789743 us; error 31 us; skew 500 ppm
I20260808 02:47:18.790119 17034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:18 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157238, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:18.793238 17034 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:47:18.794647 17034 webserver.cc:460] Webserver started at http://127.25.137.254:43819/ using document root <none> and password file <none>
I20260808 02:47:18.795282 17034 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:18.795389 17034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:18.800866 17034 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:47:18.803797 17057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:18.805243 17034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:47:18.805423 17034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "fb3fb0cb9614444ca57c3d0959d15b50"
format_stamp: "Formatted at 2026-08-08 02:47:12 on dist-test-slave-k5rr"
server_key: "1c64a7e11a492ec312b743bd521df961"
server_key_iv: "d7a7138aeff53c6eddb52d012068bfc9"
server_key_version: "encryptionkey@0"
I20260808 02:47:18.806453 17034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:18.825688 17034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:18.829919 17034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:18.830199 17034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:18.849243 17034 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:43729
I20260808 02:47:18.849264 17152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:43729 every 8 connection(s)
I20260808 02:47:18.850837 17034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:47:18.851655 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 17034
I20260808 02:47:18.852624 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:33067
--local_ip_for_outbound_sockets=127.25.137.193
--tserver_master_addrs=127.25.137.254:43729
--webserver_port=40805
--webserver_interface=127.25.137.193
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:47:18.858170 17153 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:47:18.861579 17153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Bootstrap starting.
I20260808 02:47:18.882938 17153 log.cc:824] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:18.900789 17153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:47:18.901445 17153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50: Bootstrap complete.
I20260808 02:47:18.910925 17153 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:18.911972 17153 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb3fb0cb9614444ca57c3d0959d15b50, State: Initialized, Role: FOLLOWER
I20260808 02:47:18.912595 17153 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:18.912864 17153 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:47:18.912997 17153 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:47:18.913200 17153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 2 FOLLOWER]: Advancing to term 3
I20260808 02:47:18.917182 17153 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:18.917717 17153 leader_election.cc:302] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb3fb0cb9614444ca57c3d0959d15b50; no voters:
I20260808 02:47:18.918370 17153 leader_election.cc:288] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 02:47:18.918555 17158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 02:47:18.919530 17158 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [term 3 LEADER]: Becoming Leader. State: Replica: fb3fb0cb9614444ca57c3d0959d15b50, State: Running, Role: LEADER
I20260808 02:47:18.920169 17158 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } }
I20260808 02:47:18.920446 17153 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:47:18.923094 17159 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } } }
I20260808 02:47:18.923225 17161 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb3fb0cb9614444ca57c3d0959d15b50. Latest consensus state: current_term: 3 leader_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3fb0cb9614444ca57c3d0959d15b50" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 43729 } } }
I20260808 02:47:18.923522 17161 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:18.923527 17159 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3fb0cb9614444ca57c3d0959d15b50 [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:18.934904 17170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:47:18.943734 17170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:47:18.945439 17170 catalog_manager.cc:1259] Loaded cluster ID: de8e7cb8501e45d0a13dcb0f592c41f1
I20260808 02:47:18.945632 17170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 02:47:18.947921 17170 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8379f2f9ad google::LogMessage::Fail() at ??:0
@ 0x7f8379f3288a google::LogMessage::SendToLog() at ??:0
@ 0x7f8379f2f4de google::LogMessage::Flush() at ??:0
@ 0x7f8379f30a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f838f962512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f838f95f976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f838f9e7610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f838f9e7439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f837cb7b345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f837cb7e91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f837cb7e799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f837cb59eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f837fe336db start_thread at ??:0
@ 0x7f8378dc961f clone at ??:0
W20260808 02:47:19.233815 17155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:19.234097 17155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:19.234192 17155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:19.241467 17155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:19.241572 17155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:19.241689 17155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:47:19.251194 17155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:33067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=40805
--tserver_master_addrs=127.25.137.254:43729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:19.252418 17155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:19.253770 17155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:19.266486 17231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:19.266677 17232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:19.266798 17235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:19.267968 17155 server_base.cc:1034] running on GCE node
I20260808 02:47:19.268607 17155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:19.269624 17155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:19.270824 17155 hybrid_clock.cc:648] HybridClock initialized: now 1786157239270754 us; error 65 us; skew 500 ppm
I20260808 02:47:19.271137 17155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:19 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157239, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:19.274313 17155 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:47:19.275763 17155 webserver.cc:460] Webserver started at http://127.25.137.193:40805/ using document root <none> and password file <none>
I20260808 02:47:19.276425 17155 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:19.276546 17155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:19.282059 17155 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:47:19.285104 17245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:19.286490 17155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:47:19.286661 17155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "49b1e360420644508f80772cd56ac08b"
format_stamp: "Formatted at 2026-08-08 02:47:13 on dist-test-slave-k5rr"
server_key: "010819d033aa166e539a440acbceb1b3"
server_key_iv: "e1c6078efe5fd2d823e49194fb7531ff"
server_key_version: "encryptionkey@0"
I20260808 02:47:19.287652 17155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:19.306133 17155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:19.310177 17155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:19.310421 17155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:19.311478 17155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:19.312811 17155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:19.312911 17155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:19.313050 17155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:19.313129 17155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:19.345139 17155 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:33067
I20260808 02:47:19.345196 17445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:33067 every 8 connection(s)
I20260808 02:47:19.346589 17155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:47:19.351595 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:19.349682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:45057)
0808 02:47:19.350421 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:19.350684 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":275,"threads_started":1}
W20260808 02:47:19.352310 17446 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:43729 (0 consecutive failures): Network error: Failed to ping master at 127.25.137.254:43729: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
I20260808 02:47:19.352455 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 17155
I20260808 02:47:19.353214 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:19.352777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:58081)
0808 02:47:19.352948 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:19.353066 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 02:47:19.353299 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:36329
--local_ip_for_outbound_sockets=127.25.137.194
--tserver_master_addrs=127.25.137.254:43729
--webserver_port=36553
--webserver_interface=127.25.137.194
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:47:19.354158 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:19.353836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:49087)
0808 02:47:19.353974 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:19.354048 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 02:47:19.354387 17446 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:47:19.688004 17451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:19.688279 17451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:19.688402 17451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:19.695793 17451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:19.695922 17451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:19.696079 17451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:47:19.705986 17451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:36329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=36553
--tserver_master_addrs=127.25.137.254:43729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:19.707262 17451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:19.708519 17451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:19.721403 17465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:19.721407 17462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:19.721407 17459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:19.722332 17451 server_base.cc:1034] running on GCE node
I20260808 02:47:19.722950 17451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:19.723884 17451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:19.725104 17451 hybrid_clock.cc:648] HybridClock initialized: now 1786157239725055 us; error 41 us; skew 500 ppm
I20260808 02:47:19.725411 17451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:19 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157239, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:19.728484 17451 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:47:19.729910 17451 webserver.cc:460] Webserver started at http://127.25.137.194:36553/ using document root <none> and password file <none>
I20260808 02:47:19.730593 17451 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:19.730696 17451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:19.735992 17451 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 02:47:19.738762 17476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:19.740118 17451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:47:19.740291 17451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "8d9d6454766345c986b8bc519d8432e5"
format_stamp: "Formatted at 2026-08-08 02:47:13 on dist-test-slave-k5rr"
server_key: "1aca603eef7e19a51969830fe22bc225"
server_key_iv: "bbb64766eb54a39d14220df977fb61cb"
server_key_version: "encryptionkey@0"
I20260808 02:47:19.741319 17451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:19.774852 17451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:19.778950 17451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:19.779178 17451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:19.780169 17451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:19.781548 17451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:19.781634 17451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:19.781716 17451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:19.781778 17451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:19.813660 17451 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:36329
I20260808 02:47:19.813714 17670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:36329 every 8 connection(s)
I20260808 02:47:19.815183 17451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:47:19.819952 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:19.818093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:46569)
0808 02:47:19.818898 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:19.819123 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":302,"threads_started":1}
W20260808 02:47:19.820645 17673 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:43729 (0 consecutive failures): Network error: Failed to ping master at 127.25.137.254:43729: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
I20260808 02:47:19.821509 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:19.821189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:35141)
0808 02:47:19.821330 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:19.821400 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 02:47:19.821923 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 17451
I20260808 02:47:19.822369 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:19.822071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:52901)
0808 02:47:19.822181 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:19.822257 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 02:47:19.822588 17673 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:47:19.822635 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:38885
--local_ip_for_outbound_sockets=127.25.137.195
--tserver_master_addrs=127.25.137.254:43729
--webserver_port=39319
--webserver_interface=127.25.137.195
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:20.156575 17678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:20.156880 17678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:20.157032 17678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:20.164433 17678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:20.164562 17678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:20.164721 17678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:47:20.175179 17678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:38885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=39319
--tserver_master_addrs=127.25.137.254:43729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:20.176527 17678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:20.177896 17678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:20.190917 17692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:20.191118 17691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:20.190959 17697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:20.192267 17678 server_base.cc:1034] running on GCE node
I20260808 02:47:20.192830 17678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:20.193842 17678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:20.195041 17678 hybrid_clock.cc:648] HybridClock initialized: now 1786157240194992 us; error 42 us; skew 500 ppm
I20260808 02:47:20.195349 17678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:20 dist-test-slave-k5rr krb5kdc[15274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157240, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:20.198516 17678 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:47:20.200024 17678 webserver.cc:460] Webserver started at http://127.25.137.195:39319/ using document root <none> and password file <none>
I20260808 02:47:20.200713 17678 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:20.200853 17678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:20.206568 17678 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 02:47:20.209551 17708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:20.210961 17678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:47:20.211179 17678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "3c8d38c776aa49099ff618715fa41bb8"
format_stamp: "Formatted at 2026-08-08 02:47:14 on dist-test-slave-k5rr"
server_key: "c2cb30737da16a77456262b3d7ec85c1"
server_key_iv: "da2e25ec69ebb5c21b8e38a740899c2d"
server_key_version: "encryptionkey@0"
I20260808 02:47:20.212212 17678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:20.240621 17678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:20.244794 17678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:20.245110 17678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:20.246162 17678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:20.247498 17678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:20.247609 17678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:20.247717 17678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:20.247788 17678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:20.280714 17678 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:38885
I20260808 02:47:20.280812 17891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:38885 every 8 connection(s)
I20260808 02:47:20.282346 17678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:47:20.287884 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:20.285671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:55795)
0808 02:47:20.286549 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:20.286961 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":301,"threads_started":1}
W20260808 02:47:20.288489 17892 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:43729 (0 consecutive failures): Network error: Failed to ping master at 127.25.137.254:43729: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
I20260808 02:47:20.289358 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:20.288995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:60563)
0808 02:47:20.289157 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:20.289227 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 02:47:20.290220 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:20.289902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:54271)
0808 02:47:20.290037 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:20.290112 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 02:47:20.290441 17892 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:47:20.291304 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 17678
I20260808 02:47:20.355919 17896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:20.355010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:49315)
0808 02:47:20.355499 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:20.355642 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":239,"threads_started":1}
I20260808 02:47:20.823879 17897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:20.823101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:33863)
0808 02:47:20.823536 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:20.823649 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":212,"threads_started":1}
I20260808 02:47:21.291914 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:21.291037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:60615)
0808 02:47:21.291527 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:21.291642 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":249,"threads_started":1}
I20260808 02:47:21.357620 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:21.356845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:38229)
0808 02:47:21.357359 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:21.357446 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":261,"threads_started":1}
I20260808 02:47:21.825532 17903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:21.824770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:47373)
0808 02:47:21.825266 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:21.825361 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":266,"threads_started":1}
I20260808 02:47:22.293715 17904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:22.292878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:36435)
0808 02:47:22.293411 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:22.293501 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":288,"threads_started":1}
I20260808 02:47:22.359316 17906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:22.358470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:55843)
0808 02:47:22.359030 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:22.359116 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":261,"threads_started":1}
I20260808 02:47:22.827062 17907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:22.826323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:42703)
0808 02:47:22.826775 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:22.826892 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":243,"threads_started":1}
I20260808 02:47:23.295445 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:23.294707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:58327)
0808 02:47:23.295155 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:23.295236 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":241,"threads_started":1}
I20260808 02:47:23.361038 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:23.360244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:50489)
0808 02:47:23.360728 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:23.360820 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":226,"threads_started":1}
I20260808 02:47:23.828598 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:23.827865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:41723)
0808 02:47:23.828328 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:23.828411 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":229,"threads_started":1}
I20260808 02:47:24.297115 17913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:24.296281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:47563)
0808 02:47:24.296800 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:24.296893 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":272,"threads_started":1}
I20260808 02:47:24.362664 17914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:24.361946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:45777)
0808 02:47:24.362411 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:24.362491 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":233,"threads_started":1}
I20260808 02:47:24.830246 17915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:24.829549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:34375)
0808 02:47:24.829997 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:24.830078 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":237,"threads_started":1}
I20260808 02:47:25.298851 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:25.298044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:38373)
0808 02:47:25.298558 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:25.298662 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":248,"threads_started":1}
I20260808 02:47:25.364310 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:25.363568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:58863)
0808 02:47:25.364029 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:25.364123 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":240,"threads_started":1}
I20260808 02:47:25.831856 17920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:25.831085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:53577)
0808 02:47:25.831571 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:25.831664 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":247,"threads_started":1}
I20260808 02:47:26.300488 17921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:26.299699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:36691)
0808 02:47:26.300198 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:26.300314 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":259,"threads_started":1}
I20260808 02:47:26.366045 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:26.365215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:54853)
0808 02:47:26.365715 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:26.365829 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":225,"threads_started":1}
I20260808 02:47:26.833671 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:26.832760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:36055)
0808 02:47:26.833352 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:26.833448 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":273,"threads_started":1}
I20260808 02:47:27.302284 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:27.301443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:47133)
0808 02:47:27.301979 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:27.302071 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":219,"threads_started":1}
I20260808 02:47:27.367601 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:27.366891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:47103)
0808 02:47:27.367347 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:27.367431 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":225,"threads_started":1}
I20260808 02:47:27.835215 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:27.834541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:37379)
0808 02:47:27.834973 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:27.835054 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":218,"threads_started":1}
I20260808 02:47:28.303833 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:28.303110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:49389)
0808 02:47:28.303545 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:28.303630 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":224,"threads_started":1}
I20260808 02:47:28.369351 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:28.368508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:50445)
0808 02:47:28.369080 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:28.369166 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":301,"threads_started":1}
I20260808 02:47:28.836866 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:28.836075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:35985)
0808 02:47:28.836570 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:28.836683 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":252,"threads_started":1}
I20260808 02:47:29.305589 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:29.304737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:46267)
0808 02:47:29.305325 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:29.305409 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":335,"threads_started":1}
I20260808 02:47:29.371133 17940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:29.370294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:55517)
0808 02:47:29.370827 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:29.370925 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":302,"threads_started":1}
I20260808 02:47:29.838780 17941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:29.837762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:39783)
0808 02:47:29.838540 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:29.838620 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":529,"threads_started":1}
I20260808 02:47:30.307166 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:30.306470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:47541)
0808 02:47:30.306924 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:30.307004 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":239,"threads_started":1}
I20260808 02:47:30.372907 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:30.372078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:47217)
0808 02:47:30.372638 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:30.372727 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":313,"threads_started":1}
I20260808 02:47:30.840451 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:30.839663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:33721)
0808 02:47:30.840166 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:30.840259 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":272,"threads_started":1}
I20260808 02:47:31.308782 17951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:31.308059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:40675)
0808 02:47:31.308509 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:31.308605 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":236,"threads_started":1}
I20260808 02:47:31.374585 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:31.373838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:34421)
0808 02:47:31.374333 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:31.374421 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":261,"threads_started":1}
I20260808 02:47:31.842072 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:31.841353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:38257)
0808 02:47:31.841823 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:31.841909 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":239,"threads_started":1}
I20260808 02:47:32.310393 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:32.309705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:43287)
0808 02:47:32.310135 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:32.310216 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":221,"threads_started":1}
I20260808 02:47:32.376359 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:32.375553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:37505)
0808 02:47:32.376041 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:32.376158 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":253,"threads_started":1}
I20260808 02:47:32.843729 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:32.842957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:45715)
0808 02:47:32.843448 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:32.843541 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":234,"threads_started":1}
I20260808 02:47:33.312093 17962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:33.311318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:55279)
0808 02:47:33.311805 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:33.311902 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":244,"threads_started":1}
I20260808 02:47:33.378088 17964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:33.377310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:52155)
0808 02:47:33.377815 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:33.377895 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":264,"threads_started":1}
I20260808 02:47:33.845413 17966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:33.844587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:37393)
0808 02:47:33.845069 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:33.845166 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":222,"threads_started":1}
I20260808 02:47:34.313716 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:34.312966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:49009)
0808 02:47:34.313439 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:34.313532 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":217,"threads_started":1}
I20260808 02:47:34.379983 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:34.379151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:51361)
0808 02:47:34.379654 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:34.379759 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":270,"threads_started":1}
I20260808 02:47:34.847242 17971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:34.846491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:54591)
0808 02:47:34.846960 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:34.847060 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":216,"threads_started":1}
I20260808 02:47:35.302163 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 17155
I20260808 02:47:35.315717 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:35.314818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:34207)
0808 02:47:35.315369 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:35.315521 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":265,"threads_started":1}
I20260808 02:47:35.382157 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:35.381047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.193:39255)
0808 02:47:35.381655 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:35.381889 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":309,"threads_started":1}
I20260808 02:47:35.849164 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:35.848280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:36783)
0808 02:47:35.848827 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:35.848926 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":261,"threads_started":1}
W20260808 02:47:36.304029 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 17155: Timed out: connection negotiation to 127.25.137.193:33067 for RPC CheckLeaks timed out after 1.000s (ON_OUTBOUND_QUEUE)
I20260808 02:47:36.304214 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 17155
I20260808 02:47:36.317574 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:36.316716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:48663)
0808 02:47:36.317309 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:36.317412 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":324,"threads_started":1}
I20260808 02:47:36.325584 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 17451
I20260808 02:47:36.850991 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:36.850195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.194:42939)
0808 02:47:36.850699 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:36.850793 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":270,"threads_started":1}
I20260808 02:47:37.319389 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:37.318544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:43729 (local address 127.25.137.195:47921)
0808 02:47:37.319085 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:37.319182 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":289,"threads_started":1}
W20260808 02:47:37.327142 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 17451: Timed out: connection negotiation to 127.25.137.194:36329 for RPC CheckLeaks timed out after 1.000s (ON_OUTBOUND_QUEUE)
I20260808 02:47:37.327281 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 17451
I20260808 02:47:37.354442 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 17678
W20260808 02:47:37.847661 17895 sasl_common.cc:323] Time spent creating new SASL kudu client: real 2.535s user 0.000s sys 0.001s
W20260808 02:47:37.850777 17895 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:47:35.303197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:33067 (local address 127.0.0.1:46914)
0808 02:47:35.304779 (+ 1582us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:35.304806 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:35.304967 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:35.307507 (+ 2540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:35.307512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:35.308681 (+ 1169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:35.308908 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:35.308918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:35.310701 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:35.310706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:35.311666 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:35.311678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:35.312704 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:35.312708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:35.312852 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:37.847832 (+2534980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:37.847863 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:37.850078 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:37.850531 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:33067: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"client-negotiator.queue_time_us":1010}
W20260808 02:47:37.851583 17895 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:47:36.326309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:36329 (local address 127.0.0.1:50368)
0808 02:47:37.851037 (+1524728us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:37.851056 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:37.851214 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:37.851395 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.194:36329: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"client-negotiator.queue_time_us":1524610}
I20260808 02:47:37.872328 17984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:37.355520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37294 (local address 127.25.137.195:38885)
0808 02:47:37.356198 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:37.356212 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:37.851958 (+495746us) server_negotiation.cc:407] Connection header received
0808 02:47:37.853679 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:37.853698 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:37.854141 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:37.854491 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:37.857768 (+ 3277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:37.859547 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:37.860887 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:37.862043 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:37.862260 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:37.865209 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:37.865218 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:37.865234 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:37.865280 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:37.868564 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:37.869375 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:37.869383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:37.869390 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:37.869602 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:37.870111 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:37.870116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:37.870120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:37.870826 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:37.871119 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:37.871607 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 02:47:37.872041 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":310,"threads_started":1}
I20260808 02:47:37.873836 17882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:37.959539 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 17678
2026-08-08T02:47:37Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (27113 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:47:38 dist-test-slave-k5rr krb5kdc[17992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:47:38 dist-test-slave-k5rr krb5kdc[17992](info): set up 2 sockets
Aug 08 02:47:38 dist-test-slave-k5rr krb5kdc[17992](info): commencing operation
krb5kdc: starting...
W20260808 02:47:39.178671 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:47:39 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:47:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:47:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:47:39.474470 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:42585
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:42585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:39.816251 18009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:39.816560 18009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:39.816680 18009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:39.823935 18009 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:47:39.824028 18009 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:47:39.824086 18009 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:47:39.824158 18009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:47:39.824224 18009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:39.824321 18009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:39.824383 18009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:47:39.824456 18009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:47:39.834333 18009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:42585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:42585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:39.835638 18009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:39.836961 18009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:39.848177 18016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:39.848323 18022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:39.848351 18019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:39.849448 18009 server_base.cc:1034] running on GCE node
I20260808 02:47:39.850183 18009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:39.852519 18009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:39.853791 18009 hybrid_clock.cc:648] HybridClock initialized: now 1786157259853691 us; error 37 us; skew 500 ppm
I20260808 02:47:39.854120 18009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:39 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157259, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:39.857236 18009 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:47:39.858575 18009 webserver.cc:460] Webserver started at http://127.25.137.254:40559/ using document root <none> and password file <none>
I20260808 02:47:39.859234 18009 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:39.859318 18009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:39.859579 18009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:39.862546 18009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9"
format_stamp: "Formatted at 2026-08-08 02:47:39 on dist-test-slave-k5rr"
server_key: "341a9157efc8df857e67882e78bf274d"
server_key_iv: "be56ebeb7e14c32de55c1a9cace64450"
server_key_version: "encryptionkey@0"
I20260808 02:47:39.863284 18009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9"
format_stamp: "Formatted at 2026-08-08 02:47:39 on dist-test-slave-k5rr"
server_key: "341a9157efc8df857e67882e78bf274d"
server_key_iv: "be56ebeb7e14c32de55c1a9cace64450"
server_key_version: "encryptionkey@0"
I20260808 02:47:39.868487 18009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 02:47:39.872077 18029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:39.873512 18009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:47:39.873661 18009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9"
format_stamp: "Formatted at 2026-08-08 02:47:39 on dist-test-slave-k5rr"
server_key: "341a9157efc8df857e67882e78bf274d"
server_key_iv: "be56ebeb7e14c32de55c1a9cace64450"
server_key_version: "encryptionkey@0"
I20260808 02:47:39.873872 18009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:39.889043 18009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:39.892787 18009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:39.893075 18009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:39.909857 18009 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:42585
I20260808 02:47:39.909883 18119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:42585 every 8 connection(s)
I20260808 02:47:39.911232 18009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:47:39.914861 18121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:39.918217 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 18009
I20260808 02:47:39.918431 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:47:39.919093 26151 external_mini_cluster.cc:1428] Setting key 1e30bb7dc5e2f5af544da20452950d67
I20260808 02:47:39.928167 18121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Bootstrap starting.
I20260808 02:47:39.931728 18121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:39.932678 18121 log.cc:824] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:39.935472 18121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: No bootstrap required, opened a new log
Aug 08 02:47:39 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:39.945205 18121 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:47:39.945714 18121 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:39.945827 18121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26e5a85104bd4a9ab1c54ff89aeb55c9, State: Initialized, Role: FOLLOWER
I20260808 02:47:39.946578 18121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:47:39.946902 18121 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:47:39.947053 18121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:47:39.947229 18121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:39.947631 18127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:39.921124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58086 (local address 127.25.137.254:42585)
0808 02:47:39.921968 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:39.921982 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:39.922032 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:47:39.923897 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:39.923926 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:39.924453 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:39.924911 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:39.926434 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:39.928374 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:39.929787 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:39.930782 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:39.930975 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:39.938503 (+ 7528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:39.938514 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:39.938532 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:39.938589 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:39.942697 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:39.943494 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:39.943503 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:39.943512 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:39.943729 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:39.944148 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:39.944153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:39.944156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:39.944932 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:39.945280 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:39.945687 (+ 407us) server_negotiation.cc:299] Negotiation successful
0808 02:47:39.946187 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":345,"threads_started":1}
I20260808 02:47:39.950268 18121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:47:39.950834 18121 leader_election.cc:302] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26e5a85104bd4a9ab1c54ff89aeb55c9; no voters:
I20260808 02:47:39.951411 18121 leader_election.cc:288] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:47:39.951546 18130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:47:39.952214 18130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 1 LEADER]: Becoming Leader. State: Replica: 26e5a85104bd4a9ab1c54ff89aeb55c9, State: Running, Role: LEADER
I20260808 02:47:39.952782 18130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:47:39.953411 18121 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:47:39.955708 18131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } } }
I20260808 02:47:39.955936 18132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26e5a85104bd4a9ab1c54ff89aeb55c9. Latest consensus state: current_term: 1 leader_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } } }
I20260808 02:47:39.956275 18131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:39.956284 18132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:47:39.957217 18140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:47:39.962641 18140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:47:39.970969 18140 catalog_manager.cc:1347] Generated new cluster ID: 821950171b7a45c3914a5f60aa5628bd
I20260808 02:47:39.971084 18140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:47:39.988257 18140 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:47:39.990192 18140 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:47:40.002260 18140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Generated new TSK 0
I20260808 02:47:40.003170 18140 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:47:40.088459 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:42585
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:40.423954 18165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:40.424222 18165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:40.424361 18165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:40.431623 18165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:40.431727 18165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:40.431967 18165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:47:40.442292 18165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:40.443639 18165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:40.444993 18165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:40.458578 18175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:40.458722 18172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:40.458675 18177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:40.459623 18165 server_base.cc:1034] running on GCE node
I20260808 02:47:40.460233 18165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:40.461208 18165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:40.462432 18165 hybrid_clock.cc:648] HybridClock initialized: now 1786157260462379 us; error 43 us; skew 500 ppm
I20260808 02:47:40.462743 18165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:40 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157260, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:40.465878 18165 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:47:40.467255 18165 webserver.cc:460] Webserver started at http://127.25.137.193:45979/ using document root <none> and password file <none>
I20260808 02:47:40.467902 18165 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:40.468020 18165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:40.468308 18165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:40.471197 18165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "cfad652e008245bfa733fa62252bf4f7"
format_stamp: "Formatted at 2026-08-08 02:47:40 on dist-test-slave-k5rr"
server_key: "4176e593ac8029ebe8f80c34b4cf518e"
server_key_iv: "ccbb8254ef44316f2dbcffcf001d9402"
server_key_version: "encryptionkey@0"
I20260808 02:47:40.471956 18165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "cfad652e008245bfa733fa62252bf4f7"
format_stamp: "Formatted at 2026-08-08 02:47:40 on dist-test-slave-k5rr"
server_key: "4176e593ac8029ebe8f80c34b4cf518e"
server_key_iv: "ccbb8254ef44316f2dbcffcf001d9402"
server_key_version: "encryptionkey@0"
I20260808 02:47:40.476971 18165 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 02:47:40.480427 18190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:40.481729 18165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:47:40.481906 18165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "cfad652e008245bfa733fa62252bf4f7"
format_stamp: "Formatted at 2026-08-08 02:47:40 on dist-test-slave-k5rr"
server_key: "4176e593ac8029ebe8f80c34b4cf518e"
server_key_iv: "ccbb8254ef44316f2dbcffcf001d9402"
server_key_version: "encryptionkey@0"
I20260808 02:47:40.482086 18165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:40.501432 18165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:40.505686 18165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:40.505977 18165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:40.506956 18165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:40.508313 18165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:40.508386 18165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:40.508481 18165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:40.508523 18165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:40.538990 18165 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:34443
I20260808 02:47:40.539067 18369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:34443 every 8 connection(s)
I20260808 02:47:40.540230 18165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:47:40.548105 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 18165
I20260808 02:47:40.548298 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:47:40.549057 26151 external_mini_cluster.cc:1428] Setting key 6b5ccfb986aa03c1c2d2261e9ee57ba4
Aug 08 02:47:40 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157260, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:40.561084 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:40.542783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:34421 (local address 127.25.137.254:42585)
0808 02:47:40.543235 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:40.543239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:40.544190 (+ 951us) server_negotiation.cc:407] Connection header received
0808 02:47:40.546110 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:40.546116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:40.546202 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:40.546304 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:40.547852 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:40.548647 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:40.550635 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:40.551177 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:40.551278 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:40.555283 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:40.555288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:40.555292 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:40.555327 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:40.557919 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:40.558825 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:40.558829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:40.558832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:40.558952 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:40.559388 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:40.559391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:40.559393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:40.559642 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:40.559752 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:40.560593 (+ 841us) server_negotiation.cc:299] Negotiation successful
0808 02:47:40.560766 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":233,"threads_started":1}
I20260808 02:47:40.562340 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:40.543302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:34421)
0808 02:47:40.544008 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:40.544050 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:40.545841 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:40.546564 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:40.546572 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:40.547071 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:40.547663 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:40.547688 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:40.548803 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:40.548808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:40.550497 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:40.550510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:40.551446 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:40.551454 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:40.551748 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:40.552427 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:40.552459 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:40.555061 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:40.558066 (+ 3005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:40.558075 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:40.558091 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:40.558711 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:40.559048 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:40.559052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:40.559055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:40.559297 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:40.559759 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:40.559766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:40.560174 (+ 408us) client_negotiation.cc:769] Sending connection context
0808 02:47:40.560470 (+ 296us) client_negotiation.cc:240] Negotiation successful
0808 02:47:40.560981 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":273,"threads_started":1}
I20260808 02:47:40.563874 18374 heartbeater.cc:347] Connected to a master server at 127.25.137.254:42585
I20260808 02:47:40.564175 18374 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:40.565043 18374 heartbeater.cc:511] Master 127.25.137.254:42585 requested a full tablet report, sending...
I20260808 02:47:40.567348 18057 ts_manager.cc:194] Registered new tserver with Master: cfad652e008245bfa733fa62252bf4f7 (127.25.137.193:34443)
I20260808 02:47:40.568810 18057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:34421
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:47:40.612118 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:42585
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:40.956768 18382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:40.957122 18382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:40.957299 18382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:40.964862 18382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:40.964984 18382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:40.965196 18382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:47:40.975239 18382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:40.976560 18382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:40.978022 18382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:40.991338 18392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:40.991299 18390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:40.991372 18395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:40.992112 18382 server_base.cc:1034] running on GCE node
I20260808 02:47:40.992789 18382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:40.993896 18382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:40.995128 18382 hybrid_clock.cc:648] HybridClock initialized: now 1786157260995076 us; error 34 us; skew 500 ppm
I20260808 02:47:40.995447 18382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:40 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157260, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:40.998807 18382 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:47:41.000402 18382 webserver.cc:460] Webserver started at http://127.25.137.194:39025/ using document root <none> and password file <none>
I20260808 02:47:41.001123 18382 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:41.001255 18382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:41.001541 18382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:41.004616 18382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "1cf4f4f3a1f040fb823e5dbad200dc94"
format_stamp: "Formatted at 2026-08-08 02:47:40 on dist-test-slave-k5rr"
server_key: "6c25a695052bc74635c4306983987233"
server_key_iv: "07390db24b0797a569c764f2a6c9f70b"
server_key_version: "encryptionkey@0"
I20260808 02:47:41.005453 18382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "1cf4f4f3a1f040fb823e5dbad200dc94"
format_stamp: "Formatted at 2026-08-08 02:47:40 on dist-test-slave-k5rr"
server_key: "6c25a695052bc74635c4306983987233"
server_key_iv: "07390db24b0797a569c764f2a6c9f70b"
server_key_version: "encryptionkey@0"
I20260808 02:47:41.010921 18382 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:47:41.014534 18411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:41.015868 18382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:47:41.016027 18382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "1cf4f4f3a1f040fb823e5dbad200dc94"
format_stamp: "Formatted at 2026-08-08 02:47:40 on dist-test-slave-k5rr"
server_key: "6c25a695052bc74635c4306983987233"
server_key_iv: "07390db24b0797a569c764f2a6c9f70b"
server_key_version: "encryptionkey@0"
I20260808 02:47:41.016213 18382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:41.051553 18382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:41.056149 18382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:41.056464 18382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:41.057605 18382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:41.059103 18382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:41.059191 18382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:41.059336 18382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:41.059396 18382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:41.092864 18382 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:33567
I20260808 02:47:41.092898 18615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:33567 every 8 connection(s)
I20260808 02:47:41.094261 18382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:47:41.100937 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 18382
I20260808 02:47:41.101171 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:47:41.101888 26151 external_mini_cluster.cc:1428] Setting key 460f8cbf2f01ed6c1fee1a43a9b25819
Aug 08 02:47:41 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157260, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:41.116432 18619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.096830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:56633 (local address 127.25.137.254:42585)
0808 02:47:41.097322 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:41.097326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:41.098234 (+ 908us) server_negotiation.cc:407] Connection header received
0808 02:47:41.100072 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:41.100078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:41.100157 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:41.100253 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:41.102382 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.103276 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.105376 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.105866 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.105984 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.110053 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:41.110060 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:41.110063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:41.110100 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:41.113075 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:41.113989 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:41.113994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:41.113996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:41.114147 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:41.114616 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:41.114619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:41.114621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:41.114914 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:41.115038 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:41.115929 (+ 891us) server_negotiation.cc:299] Negotiation successful
0808 02:47:41.116117 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":255,"threads_started":1}
I20260808 02:47:41.117617 18621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.097370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:56633)
0808 02:47:41.098080 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:41.098117 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:41.099801 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:41.100551 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:41.100563 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:41.101357 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:41.102160 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.102188 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.103441 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.103447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.105201 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.105217 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.106107 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.106115 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.106411 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.107103 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:41.107136 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:41.109786 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:41.113255 (+ 3469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:41.113264 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:41.113286 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:41.113845 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:41.114255 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:41.114259 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:41.114263 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:41.114512 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:41.115039 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:41.115047 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:41.115484 (+ 437us) client_negotiation.cc:769] Sending connection context
0808 02:47:41.115775 (+ 291us) client_negotiation.cc:240] Negotiation successful
0808 02:47:41.116306 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":334,"threads_started":1}
I20260808 02:47:41.119179 18616 heartbeater.cc:347] Connected to a master server at 127.25.137.254:42585
I20260808 02:47:41.119472 18616 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:41.120208 18616 heartbeater.cc:511] Master 127.25.137.254:42585 requested a full tablet report, sending...
I20260808 02:47:41.121671 18057 ts_manager.cc:194] Registered new tserver with Master: 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567)
I20260808 02:47:41.122491 18057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:56633
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:47:41.168015 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:42585
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:47:41.508852 18629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:47:41.509187 18629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:47:41.509352 18629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:47:41.516837 18629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:47:41.516973 18629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:47:41.517144 18629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:47:41.527603 18629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:47:41.528887 18629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:47:41.530269 18629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:47:41.542817 18642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:41.543334 18647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:47:41.542882 18641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:47:41.544492 18629 server_base.cc:1034] running on GCE node
I20260808 02:47:41.545189 18629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:47:41.546133 18629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:47:41.547367 18629 hybrid_clock.cc:648] HybridClock initialized: now 1786157261547300 us; error 55 us; skew 500 ppm
I20260808 02:47:41.547686 18629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:47:41 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157261, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:47:41.550899 18629 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:47:41.552322 18629 webserver.cc:460] Webserver started at http://127.25.137.195:35039/ using document root <none> and password file <none>
I20260808 02:47:41.553069 18629 fs_manager.cc:359] Metadata directory not provided
I20260808 02:47:41.553193 18629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:47:41.553481 18629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:47:41.556478 18629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "d16e9369eba84097ba56d4e21ca9b2ec"
format_stamp: "Formatted at 2026-08-08 02:47:41 on dist-test-slave-k5rr"
server_key: "3b2229831af9ea285303d7fc73680907"
server_key_iv: "00d55b46b18dec95eb4d85b90b1b4fc7"
server_key_version: "encryptionkey@0"
I20260808 02:47:41.557296 18629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "d16e9369eba84097ba56d4e21ca9b2ec"
format_stamp: "Formatted at 2026-08-08 02:47:41 on dist-test-slave-k5rr"
server_key: "3b2229831af9ea285303d7fc73680907"
server_key_iv: "00d55b46b18dec95eb4d85b90b1b4fc7"
server_key_version: "encryptionkey@0"
I20260808 02:47:41.562567 18629 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:47:41.566179 18663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:41.567536 18629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:47:41.567703 18629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "d16e9369eba84097ba56d4e21ca9b2ec"
format_stamp: "Formatted at 2026-08-08 02:47:41 on dist-test-slave-k5rr"
server_key: "3b2229831af9ea285303d7fc73680907"
server_key_iv: "00d55b46b18dec95eb4d85b90b1b4fc7"
server_key_version: "encryptionkey@0"
I20260808 02:47:41.567903 18629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:47:41.572113 18374 heartbeater.cc:503] Master 127.25.137.254:42585 was elected leader, sending a full tablet report...
I20260808 02:47:41.591526 18629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:47:41.595664 18629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:47:41.595954 18629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:47:41.596982 18629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:47:41.598328 18629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:47:41.598402 18629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:41.598486 18629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:47:41.598529 18629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:47:41.630183 18629 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:43369
I20260808 02:47:41.630234 18853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:43369 every 8 connection(s)
I20260808 02:47:41.631450 18629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:47:41.637529 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 18629
I20260808 02:47:41.637684 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:47:41.638130 26151 external_mini_cluster.cc:1428] Setting key 110803a930d3c0027929fdd65942232d
Aug 08 02:47:41 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157261, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:47:41.652092 18858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.634011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:58605 (local address 127.25.137.254:42585)
0808 02:47:41.634454 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:41.634459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:41.635846 (+ 1387us) server_negotiation.cc:407] Connection header received
0808 02:47:41.637851 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:41.637857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:41.637937 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:41.638035 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:41.639849 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.640681 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.642263 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.642659 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.642768 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.646500 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:41.646505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:41.646508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:41.646547 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:41.649029 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:41.649734 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:41.649738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:41.649740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:41.649874 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:41.650283 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:41.650287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:41.650290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:41.650554 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:41.650698 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:41.651370 (+ 672us) server_negotiation.cc:299] Negotiation successful
0808 02:47:41.651608 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":212,"threads_started":1}
I20260808 02:47:41.653192 18859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.634468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:58605)
0808 02:47:41.635674 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:41.635719 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:41.637538 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:41.638306 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:41.638315 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:41.638954 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:47:41.639684 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.639705 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.640834 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.640838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.642110 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.642124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.642883 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.642891 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.643105 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.643722 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:47:41.643745 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:47:41.646309 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:47:41.649162 (+ 2853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:41.649171 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:41.649188 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:41.649608 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:41.649978 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:47:41.649983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:47:41.649986 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:47:41.650165 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:47:41.650682 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:47:41.650690 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:47:41.651021 (+ 331us) client_negotiation.cc:769] Sending connection context
0808 02:47:41.651247 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 02:47:41.651806 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"thread_start_us":753,"threads_started":1}
I20260808 02:47:41.654481 18856 heartbeater.cc:347] Connected to a master server at 127.25.137.254:42585
I20260808 02:47:41.654838 18856 heartbeater.cc:464] Registering TS with master...
I20260808 02:47:41.655551 18856 heartbeater.cc:511] Master 127.25.137.254:42585 requested a full tablet report, sending...
I20260808 02:47:41.656922 18057 ts_manager.cc:194] Registered new tserver with Master: d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369)
I20260808 02:47:41.657783 18057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:58605
I20260808 02:47:41.665726 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:47:41.683238 18858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.670294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58088 (local address 127.25.137.254:42585)
0808 02:47:41.670502 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:41.670509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:41.670901 (+ 392us) server_negotiation.cc:407] Connection header received
0808 02:47:41.671040 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:41.671044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:41.671121 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:41.671199 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:41.672779 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.673557 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.674967 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.675360 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.675458 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.678224 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:41.678229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:41.678233 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:41.678262 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:41.680748 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:41.681445 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:41.681448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:41.681451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:41.681577 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:41.681919 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:41.681922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:41.681924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:41.682239 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:41.682428 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:41.682763 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 02:47:41.682955 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:47:41.687454 18057 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:47:41.694307 18057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:47:41.716765 18877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.707072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.25.137.193:34443)
0808 02:47:41.707790 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:41.707800 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:41.708646 (+ 846us) server_negotiation.cc:407] Connection header received
0808 02:47:41.709091 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:41.709098 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:41.709272 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:41.709405 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:41.710589 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.711847 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.714580 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.715593 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.715717 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.715835 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:41.716293 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 02:47:41.716447 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":630,"threads_started":1}
I20260808 02:47:41.716890 18880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.707504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34443 (local address 127.0.0.1:46610)
0808 02:47:41.708542 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:41.708560 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:41.708799 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:41.709731 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:41.709737 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:41.709765 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:41.710378 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.710395 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.711993 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.711996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.714394 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.714409 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.715963 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.715967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.716086 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.716116 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 02:47:41.716222 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 02:47:41.716426 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":922,"mutex_wait_us":67,"thread_start_us":745,"threads_started":1}
I20260808 02:47:41.717340 18879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.707551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:33567 (local address 127.0.0.1:53358)
0808 02:47:41.709059 (+ 1508us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:41.709076 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:41.709193 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:41.710384 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:41.710388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:41.710416 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:41.710870 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.710882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.712779 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.712783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.714583 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.714597 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.716406 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.716412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.716599 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.716634 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:47:41.716704 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:47:41.716841 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1342,"mutex_wait_us":1,"thread_start_us":746,"threads_started":1}
I20260808 02:47:41.717473 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.706491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:43369 (local address 127.0.0.1:55956)
0808 02:47:41.708451 (+ 1960us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:41.708491 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:41.709344 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:41.709949 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:41.709954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:41.709980 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:41.710450 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.710463 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.712451 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.712456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.714737 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.714751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.716679 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.716685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.716865 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.716896 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:47:41.716987 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 02:47:41.717139 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1682,"mutex_wait_us":56,"thread_start_us":2327,"threads_started":1}
I20260808 02:47:41.717947 18876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.706728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55956 (local address 127.25.137.195:43369)
0808 02:47:41.707479 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:41.707489 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:41.708593 (+ 1104us) server_negotiation.cc:407] Connection header received
0808 02:47:41.709512 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:41.709519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:41.709728 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:41.709923 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:41.710711 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.712301 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.714882 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.716180 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.716373 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.716534 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:41.717484 (+ 950us) server_negotiation.cc:299] Negotiation successful
0808 02:47:41.717642 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":385,"threads_started":1}
I20260808 02:47:41.718438 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.708148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53358 (local address 127.25.137.194:33567)
0808 02:47:41.709818 (+ 1670us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:41.709826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:41.709847 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:47:41.709937 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:41.709944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:41.710164 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:41.711073 (+ 909us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:41.711287 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.712649 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.714770 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.716179 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.717745 (+ 1566us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.717897 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:41.717961 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 02:47:41.718126 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1538,"thread_start_us":1407,"threads_started":1}
I20260808 02:47:41.721450 18491 tablet_service.cc:1467] Processing CreateTablet for tablet f7f07795ed3e4c5aaaf58541478c520e (DEFAULT_TABLE table=test-table [id=d290c1f770bd40e491d5435a3ac5c47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:41.721238 18268 tablet_service.cc:1467] Processing CreateTablet for tablet f7f07795ed3e4c5aaaf58541478c520e (DEFAULT_TABLE table=test-table [id=d290c1f770bd40e491d5435a3ac5c47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:41.721638 18752 tablet_service.cc:1467] Processing CreateTablet for tablet f7f07795ed3e4c5aaaf58541478c520e (DEFAULT_TABLE table=test-table [id=d290c1f770bd40e491d5435a3ac5c47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:47:41.722579 18491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7f07795ed3e4c5aaaf58541478c520e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:41.722867 18752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7f07795ed3e4c5aaaf58541478c520e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:41.723078 18268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7f07795ed3e4c5aaaf58541478c520e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:47:41.733860 18888 tablet_bootstrap.cc:492] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: Bootstrap starting.
I20260808 02:47:41.736723 18890 tablet_bootstrap.cc:492] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: Bootstrap starting.
I20260808 02:47:41.738279 18892 tablet_bootstrap.cc:492] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: Bootstrap starting.
I20260808 02:47:41.739449 18888 tablet_bootstrap.cc:654] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:41.740135 18890 tablet_bootstrap.cc:654] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:41.740760 18888 log.cc:824] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:41.741045 18892 tablet_bootstrap.cc:654] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: Neither blocks nor log segments found. Creating new log.
I20260808 02:47:41.741370 18890 log.cc:824] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:41.742161 18892 log.cc:824] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: Log is configured to *not* fsync() on all Append() calls
I20260808 02:47:41.744006 18888 tablet_bootstrap.cc:492] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: No bootstrap required, opened a new log
I20260808 02:47:41.744138 18890 tablet_bootstrap.cc:492] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: No bootstrap required, opened a new log
I20260808 02:47:41.744288 18888 ts_tablet_manager.cc:1397] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260808 02:47:41.744374 18890 ts_tablet_manager.cc:1397] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 02:47:41.744448 18892 tablet_bootstrap.cc:492] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: No bootstrap required, opened a new log
I20260808 02:47:41.744664 18892 ts_tablet_manager.cc:1397] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260808 02:47:41.751749 18892 raft_consensus.cc:348] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } } peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } }
I20260808 02:47:41.752094 18892 raft_consensus.cc:374] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:41.752219 18892 raft_consensus.cc:729] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d16e9369eba84097ba56d4e21ca9b2ec, State: Initialized, Role: FOLLOWER
I20260808 02:47:41.752739 18892 consensus_queue.cc:260] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } } peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } }
I20260808 02:47:41.753790 18856 heartbeater.cc:503] Master 127.25.137.254:42585 was elected leader, sending a full tablet report...
I20260808 02:47:41.754290 18892 ts_tablet_manager.cc:1428] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 02:47:41.754603 18888 raft_consensus.cc:348] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } } peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } }
I20260808 02:47:41.755028 18888 raft_consensus.cc:374] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:41.755162 18888 raft_consensus.cc:729] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cf4f4f3a1f040fb823e5dbad200dc94, State: Initialized, Role: FOLLOWER
I20260808 02:47:41.755309 18890 raft_consensus.cc:348] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } } peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } }
I20260808 02:47:41.755673 18890 raft_consensus.cc:374] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:47:41.755767 18890 raft_consensus.cc:729] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfad652e008245bfa733fa62252bf4f7, State: Initialized, Role: FOLLOWER
I20260808 02:47:41.755755 18888 consensus_queue.cc:260] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } } peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } }
I20260808 02:47:41.756263 18890 consensus_queue.cc:260] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } } peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } }
I20260808 02:47:41.757596 18888 ts_tablet_manager.cc:1428] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260808 02:47:41.758054 18616 heartbeater.cc:503] Master 127.25.137.254:42585 was elected leader, sending a full tablet report...
I20260808 02:47:41.758852 18890 ts_tablet_manager.cc:1428] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
W20260808 02:47:41.792766 18376 tablet.cc:2369] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:47:41.846477 18617 tablet.cc:2369] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:47:41.883054 18857 tablet.cc:2369] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:47:41.924777 18904 raft_consensus.cc:482] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:47:41.925127 18904 raft_consensus.cc:504] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } } peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } }
I20260808 02:47:41.926817 18904 leader_election.cc:288] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369), 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567)
I20260808 02:47:41.935834 18876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.927313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:41671 (local address 127.25.137.195:43369)
0808 02:47:41.927502 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:41.927508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:41.928280 (+ 772us) server_negotiation.cc:407] Connection header received
0808 02:47:41.928430 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:41.928436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:41.928516 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:41.928608 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:41.929375 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.930524 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.933575 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.934779 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.934943 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.935017 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:41.935447 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 02:47:41.935578 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:47:41.935874 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.927289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:47199 (local address 127.25.137.194:33567)
0808 02:47:41.927503 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:41.927509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:41.928201 (+ 692us) server_negotiation.cc:407] Connection header received
0808 02:47:41.928432 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:41.928437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:41.928516 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:41.928612 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:47:41.929442 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.930553 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.933565 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.934846 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.934975 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.935038 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:41.935448 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 02:47:41.935578 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:47:41.936035 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.927056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:33567 (local address 127.25.137.193:47199)
0808 02:47:41.928074 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:41.928098 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:41.928275 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:41.928640 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:41.928645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:41.928682 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:41.929294 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.929315 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.930666 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.930670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.933403 (+ 2733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.933419 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.935060 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.935066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.935249 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.935306 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 02:47:41.935397 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 02:47:41.935552 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":210,"spinlock_wait_cycles":29184,"thread_start_us":629,"threads_started":1}
I20260808 02:47:41.936324 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:41.927073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:43369 (local address 127.25.137.193:41671)
0808 02:47:41.928090 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0808 02:47:41.928103 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:47:41.928275 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:47:41.928640 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:47:41.928645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:47:41.928680 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:47:41.929204 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.929220 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.930665 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.930670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.933410 (+ 2740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:47:41.933420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:47:41.935032 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:47:41.935042 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:47:41.935243 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:41.935308 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 02:47:41.935401 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 02:47:41.935552 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"mutex_wait_us":216,"spinlock_wait_cycles":32768,"thread_start_us":660,"threads_started":1}
I20260808 02:47:41.937067 18519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f07795ed3e4c5aaaf58541478c520e" candidate_uuid: "cfad652e008245bfa733fa62252bf4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" is_pre_election: true
I20260808 02:47:41.937361 18783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f07795ed3e4c5aaaf58541478c520e" candidate_uuid: "cfad652e008245bfa733fa62252bf4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" is_pre_election: true
I20260808 02:47:41.937697 18519 raft_consensus.cc:2454] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfad652e008245bfa733fa62252bf4f7 in term 0.
I20260808 02:47:41.937891 18783 raft_consensus.cc:2454] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfad652e008245bfa733fa62252bf4f7 in term 0.
I20260808 02:47:41.938398 18196 leader_election.cc:302] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cf4f4f3a1f040fb823e5dbad200dc94, cfad652e008245bfa733fa62252bf4f7; no voters:
I20260808 02:47:41.938750 18904 raft_consensus.cc:2781] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:47:41.938897 18904 raft_consensus.cc:482] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:47:41.939003 18904 raft_consensus.cc:3037] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:41.941151 18904 raft_consensus.cc:504] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } } peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } }
I20260808 02:47:41.941718 18904 leader_election.cc:288] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 1 election: Requested vote from peers d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369), 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567)
I20260808 02:47:41.942139 18783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f07795ed3e4c5aaaf58541478c520e" candidate_uuid: "cfad652e008245bfa733fa62252bf4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d16e9369eba84097ba56d4e21ca9b2ec"
I20260808 02:47:41.942189 18519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f07795ed3e4c5aaaf58541478c520e" candidate_uuid: "cfad652e008245bfa733fa62252bf4f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94"
I20260808 02:47:41.942312 18783 raft_consensus.cc:3037] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:41.942387 18519 raft_consensus.cc:3037] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:47:41.944787 18783 raft_consensus.cc:2454] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfad652e008245bfa733fa62252bf4f7 in term 1.
I20260808 02:47:41.944929 18519 raft_consensus.cc:2454] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfad652e008245bfa733fa62252bf4f7 in term 1.
I20260808 02:47:41.945180 18193 leader_election.cc:302] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cfad652e008245bfa733fa62252bf4f7, d16e9369eba84097ba56d4e21ca9b2ec; no voters:
I20260808 02:47:41.945472 18904 raft_consensus.cc:2781] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:47:41.945856 18904 raft_consensus.cc:686] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 1 LEADER]: Becoming Leader. State: Replica: cfad652e008245bfa733fa62252bf4f7, State: Running, Role: LEADER
I20260808 02:47:41.946283 18904 consensus_queue.cc:237] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } } peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } }
I20260808 02:47:41.950951 18057 catalog_manager.cc:5495] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to cfad652e008245bfa733fa62252bf4f7 (127.25.137.193). New cstate: current_term: 1 leader_uuid: "cfad652e008245bfa733fa62252bf4f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } health_report { overall_health: UNKNOWN } } }
I20260808 02:47:42.013934 18877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:42.008443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46614 (local address 127.25.137.193:34443)
0808 02:47:42.008656 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:42.008662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:42.008682 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:47:42.008796 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:42.008800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:42.008879 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:42.009031 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:47:42.009451 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:42.010190 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:42.011625 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:42.012049 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:42.012157 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:42.012695 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:47:42.013355 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:47:42.013510 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:42.013590 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 02:47:42.013662 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:47:42.021917 18519 raft_consensus.cc:1261] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 1 FOLLOWER]: Refusing update from remote peer cfad652e008245bfa733fa62252bf4f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:47:42.021943 18783 raft_consensus.cc:1261] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec [term 1 FOLLOWER]: Refusing update from remote peer cfad652e008245bfa733fa62252bf4f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:47:42.022840 18904 consensus_queue.cc:1035] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:47:42.023118 18911 consensus_queue.cc:1035] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:47:42.046626 18053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:47:42.046947 18053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:47:42.050571 18053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Sending DeleteTablet for 3 replicas of tablet f7f07795ed3e4c5aaaf58541478c520e
I20260808 02:47:42.051717 18491 tablet_service.cc:1514] Processing DeleteTablet for tablet f7f07795ed3e4c5aaaf58541478c520e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:42 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:53358
I20260808 02:47:42.051690 18752 tablet_service.cc:1514] Processing DeleteTablet for tablet f7f07795ed3e4c5aaaf58541478c520e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:42 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:55956
I20260808 02:47:42.052323 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 18165
I20260808 02:47:42.052632 18930 tablet_replica.cc:330] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: stopping tablet replica
I20260808 02:47:42.053048 18931 tablet_replica.cc:330] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: stopping tablet replica
I20260808 02:47:42.053117 18930 raft_consensus.cc:2229] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:47:42.053344 18931 raft_consensus.cc:2229] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:47:42.053452 18268 tablet_service.cc:1514] Processing DeleteTablet for tablet f7f07795ed3e4c5aaaf58541478c520e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:47:42 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:46610
I20260808 02:47:42.053629 18930 raft_consensus.cc:2258] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:42.053753 18931 raft_consensus.cc:2258] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:42.056003 18930 ts_tablet_manager.cc:1905] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:42.056008 18931 ts_tablet_manager.cc:1905] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:42.061578 18934 tablet_replica.cc:330] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: stopping tablet replica
I20260808 02:47:42.061997 18934 raft_consensus.cc:2229] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:47:42.062785 18934 raft_consensus.cc:2258] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:47:42.063506 18931 ts_tablet_manager.cc:1918] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:42.063674 18931 log.cc:1196] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/f7f07795ed3e4c5aaaf58541478c520e
I20260808 02:47:42.064208 18931 ts_tablet_manager.cc:1939] T f7f07795ed3e4c5aaaf58541478c520e P 1cf4f4f3a1f040fb823e5dbad200dc94: Deleting consensus metadata
I20260808 02:47:42.065037 18934 ts_tablet_manager.cc:1905] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:47:42.066116 18036 catalog_manager.cc:4841] TS 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567): tablet f7f07795ed3e4c5aaaf58541478c520e (table test-table [id=d290c1f770bd40e491d5435a3ac5c47f]) successfully deleted
I20260808 02:47:42.068708 18930 ts_tablet_manager.cc:1918] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:42.068885 18930 log.cc:1196] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/f7f07795ed3e4c5aaaf58541478c520e
I20260808 02:47:42.069438 18930 ts_tablet_manager.cc:1939] T f7f07795ed3e4c5aaaf58541478c520e P d16e9369eba84097ba56d4e21ca9b2ec: Deleting consensus metadata
I20260808 02:47:42.071184 18030 catalog_manager.cc:4841] TS d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369): tablet f7f07795ed3e4c5aaaf58541478c520e (table test-table [id=d290c1f770bd40e491d5435a3ac5c47f]) successfully deleted
I20260808 02:47:42.072921 18934 ts_tablet_manager.cc:1918] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:47:42.073099 18934 log.cc:1196] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/f7f07795ed3e4c5aaaf58541478c520e
I20260808 02:47:42.073649 18934 ts_tablet_manager.cc:1939] T f7f07795ed3e4c5aaaf58541478c520e P cfad652e008245bfa733fa62252bf4f7: Deleting consensus metadata
I20260808 02:47:42.075264 18033 catalog_manager.cc:4841] TS cfad652e008245bfa733fa62252bf4f7 (127.25.137.193:34443): tablet f7f07795ed3e4c5aaaf58541478c520e (table test-table [id=d290c1f770bd40e491d5435a3ac5c47f]) successfully deleted
Aug 08 02:47:42 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:47:42.087033 18877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:42.053990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46630 (local address 127.25.137.193:34443)
0808 02:47:42.054168 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:42.054175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:42.065082 (+ 10907us) server_negotiation.cc:407] Connection header received
0808 02:47:42.065660 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:42.065665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:42.065759 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:42.065848 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:42.067572 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:42.068794 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:42.074994 (+ 6200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:42.075723 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:42.075868 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:42.081116 (+ 5248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:42.081128 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:42.081143 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:42.081175 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:42.084056 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:42.084870 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:42.084877 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:42.084879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:42.085082 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:42.085553 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:42.085557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:42.085559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:42.085900 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:42.086108 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:42.086515 (+ 407us) server_negotiation.cc:299] Negotiation successful
0808 02:47:42.086726 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:47:42.087738 18361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:42.187041 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 18165
I20260808 02:47:42.211161 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 18382
Aug 08 02:47:42 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:47:42.226289 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:42.211967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.25.137.194:33567)
0808 02:47:42.212149 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:42.212154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:42.212177 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:47:42.212391 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:42.212395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:42.212469 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:42.212552 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:42.213914 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:42.214685 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:42.215998 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:42.216390 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:42.216500 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:42.220855 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:42.220871 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:42.220886 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:42.220921 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:42.223552 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:42.224300 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:42.224308 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:42.224310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:42.224447 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:42.224750 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:42.224754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:42.224758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:42.225217 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:42.225483 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:42.225782 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 02:47:42.225949 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:47:42.226926 18600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:42.316004 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 18382
I20260808 02:47:42.340418 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 18629
Aug 08 02:47:42 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:47:42.355917 18876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:47:42.341211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.25.137.195:43369)
0808 02:47:42.341409 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 02:47:42.341414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:47:42.341462 (+ 48us) server_negotiation.cc:407] Connection header received
0808 02:47:42.341630 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:47:42.341634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:47:42.341703 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:47:42.341849 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:47:42.343199 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:42.343962 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:42.345581 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:47:42.346017 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:47:42.346117 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:47:42.350332 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:47:42.350350 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:47:42.350367 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:47:42.350403 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:47:42.353233 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:42.353958 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:42.353966 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:42.353969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:42.354134 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:47:42.354521 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:47:42.354525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:47:42.354527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:47:42.354914 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:47:42.355140 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 02:47:42.355358 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 02:47:42.355532 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:47:42.356539 18849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:42.443857 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 18629
I20260808 02:47:42.467126 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 18009
I20260808 02:47:42.468322 18112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:47:42.579712 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 18009
I20260808 02:48:02.608095 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:42585
--webserver_interface=127.25.137.254
--webserver_port=40559
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:42585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:02.954082 18942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:02.954418 18942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:02.954578 18942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:02.962224 18942 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:48:02.962347 18942 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:48:02.962419 18942 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:48:02.962499 18942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:48:02.962586 18942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:02.962693 18942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:02.962776 18942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:48:02.962863 18942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:48:02.973278 18942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:42585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:42585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:02.974599 18942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:02.976245 18942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:03.001071 18953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:03.001123 18957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:03.001101 18952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:03.001962 18942 server_base.cc:1034] running on GCE node
I20260808 02:48:03.002894 18942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:03.004525 18942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:03.005777 18942 hybrid_clock.cc:648] HybridClock initialized: now 1786157283005686 us; error 52 us; skew 500 ppm
I20260808 02:48:03.006196 18942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:03 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157283, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:03.013499 18942 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:48:03.015281 18942 webserver.cc:460] Webserver started at http://127.25.137.254:40559/ using document root <none> and password file <none>
I20260808 02:48:03.016091 18942 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:03.016222 18942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:03.022609 18942 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:48:03.025731 18966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:03.027206 18942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:48:03.027411 18942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9"
format_stamp: "Formatted at 2026-08-08 02:47:39 on dist-test-slave-k5rr"
server_key: "341a9157efc8df857e67882e78bf274d"
server_key_iv: "be56ebeb7e14c32de55c1a9cace64450"
server_key_version: "encryptionkey@0"
I20260808 02:48:03.028550 18942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:03.051656 18942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:03.056209 18942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:03.056501 18942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:03.075171 18942 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:42585
I20260808 02:48:03.075219 19044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:42585 every 8 connection(s)
I20260808 02:48:03.076763 18942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:48:03.079295 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 18942
I20260808 02:48:03.080317 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:34443
--local_ip_for_outbound_sockets=127.25.137.193
--tserver_master_addrs=127.25.137.254:42585
--webserver_port=45979
--webserver_interface=127.25.137.193
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:48:03.086057 19045 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:48:03.090701 19045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Bootstrap starting.
I20260808 02:48:03.111644 19045 log.cc:824] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:03.124217 19045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:48:03.124750 19045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Bootstrap complete.
I20260808 02:48:03.134397 19045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:48:03.135275 19045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26e5a85104bd4a9ab1c54ff89aeb55c9, State: Initialized, Role: FOLLOWER
I20260808 02:48:03.135958 19045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:48:03.136209 19045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:48:03.136337 19045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:48:03.136502 19045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:48:03.140667 19045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:48:03.141230 19045 leader_election.cc:302] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26e5a85104bd4a9ab1c54ff89aeb55c9; no voters:
I20260808 02:48:03.141903 19045 leader_election.cc:288] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:48:03.142050 19054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:48:03.143100 19054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 2 LEADER]: Becoming Leader. State: Replica: 26e5a85104bd4a9ab1c54ff89aeb55c9, State: Running, Role: LEADER
I20260808 02:48:03.143637 19054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:48:03.144444 19045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:48:03.146569 19059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26e5a85104bd4a9ab1c54ff89aeb55c9. Latest consensus state: current_term: 2 leader_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } } }
I20260808 02:48:03.146997 19059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:03.147354 19058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } } }
I20260808 02:48:03.147775 19058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:03.156147 19067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:48:03.168864 19067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:48:03.170452 19067 catalog_manager.cc:1259] Loaded cluster ID: 821950171b7a45c3914a5f60aa5628bd
I20260808 02:48:03.170626 19067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:48:03.174924 19067 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:48:03.176545 19067 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Loaded TSK: 0
I20260808 02:48:03.223351 19067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Generated new TSK 1
I20260808 02:48:03.223817 19067 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:48:03.443972 19047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:03.444259 19047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:03.444408 19047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:03.451802 19047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:03.451926 19047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:03.452072 19047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:48:03.461751 19047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:34443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=45979
--tserver_master_addrs=127.25.137.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:03.463013 19047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:03.464319 19047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:03.477890 19088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:03.478960 19091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:03.480239 19096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:03.486328 19047 server_base.cc:1034] running on GCE node
I20260808 02:48:03.487048 19047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:03.488029 19047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:03.489238 19047 hybrid_clock.cc:648] HybridClock initialized: now 1786157283489192 us; error 37 us; skew 500 ppm
I20260808 02:48:03.489593 19047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:03 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157283, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:03.492692 19047 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:48:03.494266 19047 webserver.cc:460] Webserver started at http://127.25.137.193:45979/ using document root <none> and password file <none>
I20260808 02:48:03.494971 19047 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:03.495162 19047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:03.500505 19047 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:48:03.503247 19111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:03.504544 19047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:48:03.504675 19047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "cfad652e008245bfa733fa62252bf4f7"
format_stamp: "Formatted at 2026-08-08 02:47:40 on dist-test-slave-k5rr"
server_key: "4176e593ac8029ebe8f80c34b4cf518e"
server_key_iv: "ccbb8254ef44316f2dbcffcf001d9402"
server_key_version: "encryptionkey@0"
I20260808 02:48:03.505646 19047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:03.522490 19047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:03.526671 19047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:03.526911 19047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:03.527909 19047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:03.529299 19047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:03.529385 19047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:03.529484 19047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:03.529556 19047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:03.570879 19047 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:34443
I20260808 02:48:03.570986 19296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:34443 every 8 connection(s)
I20260808 02:48:03.572391 19047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:48:03.573320 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 19047
I20260808 02:48:03.574072 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:33567
--local_ip_for_outbound_sockets=127.25.137.194
--tserver_master_addrs=127.25.137.254:42585
--webserver_port=39025
--webserver_interface=127.25.137.194
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:48:03 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157283, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:48:03.598219 19304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:03.575791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:40091)
0808 02:48:03.576697 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:03.576733 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:03.578871 (+ 2138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:03.580488 (+ 1617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:03.580497 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:03.581148 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:03.581796 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:03.581817 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:03.583951 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:03.583955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:03.585577 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:03.585591 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:03.586905 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:03.586912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:03.587181 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:03.588039 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:48:03.588065 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:03.590436 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:03.593857 (+ 3421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:03.593866 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:03.593882 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:03.594305 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:03.594750 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:03.594755 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:03.594758 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:03.594951 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:03.595733 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:03.595741 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:03.596059 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 02:48:03.596294 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 02:48:03.596869 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":314,"threads_started":1}
I20260808 02:48:03.598217 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:03.575459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:40091 (local address 127.25.137.254:42585)
0808 02:48:03.576139 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:03.576152 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:03.576876 (+ 724us) server_negotiation.cc:407] Connection header received
0808 02:48:03.579487 (+ 2611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:03.579505 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:03.579945 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:03.580336 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:03.582061 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:03.583789 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:03.585722 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:03.586667 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:03.586849 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:03.590755 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:03.590764 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:03.590779 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:03.590820 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:03.593710 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:03.594448 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:03.594455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:03.594461 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:03.594634 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:03.595074 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:03.595077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:03.595079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:03.595570 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:03.595753 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:03.596568 (+ 815us) server_negotiation.cc:299] Negotiation successful
0808 02:48:03.596967 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":263,"threads_started":1}
I20260808 02:48:03.601337 19299 heartbeater.cc:347] Connected to a master server at 127.25.137.254:42585
I20260808 02:48:03.601675 19299 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:03.602437 19299 heartbeater.cc:511] Master 127.25.137.254:42585 requested a full tablet report, sending...
I20260808 02:48:03.604579 18990 ts_manager.cc:194] Registered new tserver with Master: cfad652e008245bfa733fa62252bf4f7 (127.25.137.193:34443)
I20260808 02:48:03.606109 18990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:40091
W20260808 02:48:03.908128 19307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:03.908427 19307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:03.908567 19307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:03.915897 19307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:03.916009 19307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:03.916154 19307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:48:03.925606 19307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:33567
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=39025
--tserver_master_addrs=127.25.137.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:03.926851 19307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:03.928104 19307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:03.940259 19316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:03.940433 19323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:03.940289 19319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:03.941357 19307 server_base.cc:1034] running on GCE node
I20260808 02:48:03.941937 19307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:03.942834 19307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:03.944056 19307 hybrid_clock.cc:648] HybridClock initialized: now 1786157283943997 us; error 47 us; skew 500 ppm
I20260808 02:48:03.944373 19307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:03 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157283, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:03.948032 19307 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:48:03.949429 19307 webserver.cc:460] Webserver started at http://127.25.137.194:39025/ using document root <none> and password file <none>
I20260808 02:48:03.950091 19307 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:03.950203 19307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:03.956084 19307 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 02:48:03.958841 19334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:03.960269 19307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:48:03.960417 19307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "1cf4f4f3a1f040fb823e5dbad200dc94"
format_stamp: "Formatted at 2026-08-08 02:47:40 on dist-test-slave-k5rr"
server_key: "6c25a695052bc74635c4306983987233"
server_key_iv: "07390db24b0797a569c764f2a6c9f70b"
server_key_version: "encryptionkey@0"
I20260808 02:48:03.961426 19307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:03.978679 19307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:03.982547 19307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:03.982771 19307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:03.983700 19307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:03.984973 19307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:03.985112 19307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:03.985200 19307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:03.985268 19307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:04.019627 19307 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:33567
I20260808 02:48:04.019687 19541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:33567 every 8 connection(s)
I20260808 02:48:04.021127 19307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:48:04.027345 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 19307
I20260808 02:48:04.028116 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:43369
--local_ip_for_outbound_sockets=127.25.137.195
--tserver_master_addrs=127.25.137.254:42585
--webserver_port=35039
--webserver_interface=127.25.137.195
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:48:04 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157283, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:48:04.042562 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.023595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:49365 (local address 127.25.137.254:42585)
0808 02:48:04.023822 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:04.023829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:04.024956 (+ 1127us) server_negotiation.cc:407] Connection header received
0808 02:48:04.026795 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:04.026800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:04.026873 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:04.026963 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:04.028989 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.029806 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.031857 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.032372 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.032467 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.036540 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:04.036545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:04.036548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:04.036585 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:04.039073 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:04.039985 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:04.039990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:04.039992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:04.040143 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:04.040606 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:04.040610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:04.040612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:04.041032 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:04.041205 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:04.041887 (+ 682us) server_negotiation.cc:299] Negotiation successful
0808 02:48:04.042172 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:48:04.043318 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.024095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:49365)
0808 02:48:04.024798 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:04.024840 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:04.026530 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:04.027252 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:04.027265 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:04.028038 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:04.028807 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.028832 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.029978 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.029983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.031707 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.031724 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.032593 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.032599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.032895 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.033616 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:48:04.033645 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:04.036301 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:04.039233 (+ 2932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:04.039242 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:04.039260 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:04.039837 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:04.040259 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:04.040263 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:04.040266 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:04.040511 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:04.041164 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:04.041171 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:04.041518 (+ 347us) client_negotiation.cc:769] Sending connection context
0808 02:48:04.041748 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 02:48:04.042174 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":285,"threads_started":1}
I20260808 02:48:04.045001 19542 heartbeater.cc:347] Connected to a master server at 127.25.137.254:42585
I20260808 02:48:04.045369 19542 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:04.046142 19542 heartbeater.cc:511] Master 127.25.137.254:42585 requested a full tablet report, sending...
I20260808 02:48:04.047725 18990 ts_manager.cc:194] Registered new tserver with Master: 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567)
I20260808 02:48:04.048573 18990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:49365
W20260808 02:48:04.372051 19553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:04.372329 19553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:04.372457 19553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:04.379778 19553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:04.379890 19553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:04.380038 19553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:48:04.389575 19553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:43369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=35039
--tserver_master_addrs=127.25.137.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:04.390830 19553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:04.392146 19553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:04.404462 19567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:04.404525 19561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:04.404503 19569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:04.405787 19553 server_base.cc:1034] running on GCE node
I20260808 02:48:04.406370 19553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:04.407261 19553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:04.408471 19553 hybrid_clock.cc:648] HybridClock initialized: now 1786157284408420 us; error 44 us; skew 500 ppm
I20260808 02:48:04.408802 19553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:04 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157284, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:04.411893 19553 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:48:04.413357 19553 webserver.cc:460] Webserver started at http://127.25.137.195:35039/ using document root <none> and password file <none>
I20260808 02:48:04.413971 19553 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:04.414094 19553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:04.419231 19553 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 02:48:04.421938 19583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:04.423229 19553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:48:04.423401 19553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "d16e9369eba84097ba56d4e21ca9b2ec"
format_stamp: "Formatted at 2026-08-08 02:47:41 on dist-test-slave-k5rr"
server_key: "3b2229831af9ea285303d7fc73680907"
server_key_iv: "00d55b46b18dec95eb4d85b90b1b4fc7"
server_key_version: "encryptionkey@0"
I20260808 02:48:04.424352 19553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:04.442490 19553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:04.446723 19553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:04.446991 19553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:04.447958 19553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:04.449321 19553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:04.449429 19553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:04.449544 19553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:04.449615 19553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:04.481659 19553 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:43369
I20260808 02:48:04.481716 19768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:43369 every 8 connection(s)
I20260808 02:48:04.483134 19553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:48:04.490937 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 19553
Aug 08 02:48:04 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157284, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:48:04.506273 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.485780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:54581 (local address 127.25.137.254:42585)
0808 02:48:04.485962 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:04.485966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:04.487402 (+ 1436us) server_negotiation.cc:407] Connection header received
0808 02:48:04.489418 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:04.489425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:04.489509 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:04.489607 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:04.491569 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.492709 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.494697 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.495431 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.495584 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.499393 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:04.499400 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:04.499405 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:04.499450 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:04.502718 (+ 3268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:04.503592 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:04.503599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:04.503603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:04.503819 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:04.504309 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:04.504313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:04.504315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:04.504684 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:04.504853 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:04.505732 (+ 879us) server_negotiation.cc:299] Negotiation successful
0808 02:48:04.505978 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:48:04.507426 19774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.486342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:54581)
0808 02:48:04.487207 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:04.487258 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:04.489110 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:04.489878 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:04.489886 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:04.490496 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:04.491342 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.491372 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.492942 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.492947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.494533 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.494546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.495657 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.495662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.496018 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.496696 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:48:04.496724 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:04.499159 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:04.502865 (+ 3706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:04.502874 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:04.502891 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:04.503455 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:04.503943 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:04.503948 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:04.503952 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:04.504189 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:04.504838 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:04.504847 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:04.505323 (+ 476us) client_negotiation.cc:769] Sending connection context
0808 02:48:04.505648 (+ 325us) client_negotiation.cc:240] Negotiation successful
0808 02:48:04.506185 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":392,"threads_started":1}
I20260808 02:48:04.509068 19770 heartbeater.cc:347] Connected to a master server at 127.25.137.254:42585
I20260808 02:48:04.509421 19770 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:04.509631 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.492035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39830 (local address 127.25.137.254:42585)
0808 02:48:04.493034 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:04.493040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:04.493394 (+ 354us) server_negotiation.cc:407] Connection header received
0808 02:48:04.493586 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:04.493591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:04.493669 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:04.493781 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:04.495835 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.496859 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.498431 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.499046 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.499172 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.502787 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:04.502793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:04.502798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:04.502837 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:04.506235 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:04.507092 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:04.507097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:04.507100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:04.507241 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:04.507684 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:04.507688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:04.507690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:04.508097 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:04.508353 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:04.508829 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 02:48:04.509068 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":861,"thread_start_us":715,"threads_started":1}
I20260808 02:48:04.510411 19770 heartbeater.cc:511] Master 127.25.137.254:42585 requested a full tablet report, sending...
I20260808 02:48:04.512064 18989 ts_manager.cc:194] Registered new tserver with Master: d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369)
I20260808 02:48:04.512935 18989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:54581
I20260808 02:48:04.524313 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:48:04.531165 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.525206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39844 (local address 127.25.137.254:42585)
0808 02:48:04.525396 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:04.525404 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:04.525880 (+ 476us) server_negotiation.cc:407] Connection header received
0808 02:48:04.526113 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:04.526118 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:04.526292 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:04.526436 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:48:04.526875 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.527644 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.529253 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.529688 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.529785 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.530261 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:48:04.530543 (+ 282us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 02:48:04.530876 (+ 333us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39844: token expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 02:48:04.531312 19775 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39844: token expired
W20260808 02:48:04.531842 18866 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.25.137.254:42585)
I20260808 02:48:04.544729 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.532348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39852 (local address 127.25.137.254:42585)
0808 02:48:04.532508 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:04.532521 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:04.532539 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:48:04.532682 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:04.532686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:04.532744 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:04.532852 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:04.534276 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.535069 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.536434 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.536881 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.536982 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.539446 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:04.539451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:04.539455 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:04.539488 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:04.542185 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:04.542863 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:04.542867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:04.542870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:04.543005 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:04.543363 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:04.543366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:04.543368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:04.543681 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:04.543892 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:04.544244 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 02:48:04.544418 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:48:04.548408 18989 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:48:04.556169 18989 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:48:04.584903 19790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.568608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47658 (local address 127.25.137.193:34443)
0808 02:48:04.570349 (+ 1741us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:04.570375 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:04.572004 (+ 1629us) server_negotiation.cc:407] Connection header received
0808 02:48:04.572137 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:04.572147 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:04.573789 (+ 1642us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:04.574012 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:04.575403 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.578768 (+ 3365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.581842 (+ 3074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.583441 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.583652 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.583866 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:04.584331 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 02:48:04.584511 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1396,"thread_start_us":1253,"threads_started":1}
I20260808 02:48:04.584924 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.568394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34443 (local address 127.0.0.1:47658)
0808 02:48:04.571520 (+ 3126us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:04.571567 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:04.571766 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:04.574400 (+ 2634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:04.574539 (+ 139us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:04.574584 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:04.575131 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.575156 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.578984 (+ 3828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.578990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.581632 (+ 2642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.581653 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.583934 (+ 2281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.583941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.584105 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.584141 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:48:04.584284 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 02:48:04.584516 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2865,"thread_start_us":2762,"threads_started":1}
I20260808 02:48:04.589180 19191 tablet_service.cc:1467] Processing CreateTablet for tablet 08136a859519471d9d999100f0c4c0bc (DEFAULT_TABLE table=test-table [id=01d2777020294c8aa7b650d1b3372fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:04.591104 19191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08136a859519471d9d999100f0c4c0bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:04.591284 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.569233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46792 (local address 127.25.137.195:43369)
0808 02:48:04.570856 (+ 1623us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:04.570862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:04.583680 (+ 12818us) server_negotiation.cc:407] Connection header received
0808 02:48:04.583888 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:04.583896 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:04.584123 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:04.584325 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:04.585387 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.586804 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.588536 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.589844 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.590031 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.590212 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:04.590748 (+ 536us) server_negotiation.cc:299] Negotiation successful
0808 02:48:04.590918 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1463,"thread_start_us":1327,"threads_started":1}
I20260808 02:48:04.591639 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.568475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:33567 (local address 127.0.0.1:36840)
0808 02:48:04.583086 (+ 14611us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:04.583105 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:04.583252 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:04.584128 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:04.584134 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:04.584159 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:04.584559 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.584572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.586456 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.586462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.588705 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.588722 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.590885 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.590891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.591053 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.591087 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:48:04.591184 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 02:48:04.591312 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14486,"thread_start_us":14892,"threads_started":1}
I20260808 02:48:04.591797 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.568968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36840 (local address 127.25.137.194:33567)
0808 02:48:04.569703 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:04.569713 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:04.583253 (+ 13540us) server_negotiation.cc:407] Connection header received
0808 02:48:04.583414 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:04.583422 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:04.583686 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:04.583888 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:04.584764 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.586308 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.588911 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.590411 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.590573 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.590733 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:04.591262 (+ 529us) server_negotiation.cc:299] Negotiation successful
0808 02:48:04.591451 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":429,"threads_started":1}
I20260808 02:48:04.592021 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.569769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:43369 (local address 127.0.0.1:46792)
0808 02:48:04.583529 (+ 13760us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:04.583555 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:04.583682 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:04.584610 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:04.584614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:04.584639 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:04.585170 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.585185 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.586941 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.586945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.588381 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.588392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.590372 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.590378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.590543 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.590577 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:48:04.591568 (+ 991us) client_negotiation.cc:240] Negotiation successful
0808 02:48:04.591704 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13598,"thread_start_us":12964,"threads_started":1}
I20260808 02:48:04.595538 19650 tablet_service.cc:1467] Processing CreateTablet for tablet 08136a859519471d9d999100f0c4c0bc (DEFAULT_TABLE table=test-table [id=01d2777020294c8aa7b650d1b3372fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:04.595544 19432 tablet_service.cc:1467] Processing CreateTablet for tablet 08136a859519471d9d999100f0c4c0bc (DEFAULT_TABLE table=test-table [id=01d2777020294c8aa7b650d1b3372fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:04.596938 19432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08136a859519471d9d999100f0c4c0bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:04.596935 19650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08136a859519471d9d999100f0c4c0bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:04.605516 19800 tablet_bootstrap.cc:492] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Bootstrap starting.
I20260808 02:48:04.608891 19800 tablet_bootstrap.cc:654] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:04.609244 19299 heartbeater.cc:503] Master 127.25.137.254:42585 was elected leader, sending a full tablet report...
I20260808 02:48:04.610339 19800 log.cc:824] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:04.615298 19805 tablet_bootstrap.cc:492] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: Bootstrap starting.
I20260808 02:48:04.617360 19800 tablet_bootstrap.cc:492] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: No bootstrap required, opened a new log
I20260808 02:48:04.617652 19800 ts_tablet_manager.cc:1397] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260808 02:48:04.619112 19805 tablet_bootstrap.cc:654] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:04.620435 19805 log.cc:824] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:04.621335 19807 tablet_bootstrap.cc:492] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: Bootstrap starting.
I20260808 02:48:04.624433 19807 tablet_bootstrap.cc:654] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:04.625627 19807 log.cc:824] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:04.627389 19805 tablet_bootstrap.cc:492] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: No bootstrap required, opened a new log
I20260808 02:48:04.627714 19805 ts_tablet_manager.cc:1397] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.011s
I20260808 02:48:04.628168 19807 tablet_bootstrap.cc:492] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: No bootstrap required, opened a new log
I20260808 02:48:04.628400 19807 ts_tablet_manager.cc:1397] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:48:04.628484 19800 raft_consensus.cc:348] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:04.628959 19800 raft_consensus.cc:374] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:04.629110 19800 raft_consensus.cc:729] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfad652e008245bfa733fa62252bf4f7, State: Initialized, Role: FOLLOWER
I20260808 02:48:04.629783 19800 consensus_queue.cc:260] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:04.632272 19800 ts_tablet_manager.cc:1428] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260808 02:48:04.636933 19807 raft_consensus.cc:348] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:04.637338 19807 raft_consensus.cc:374] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:04.637436 19807 raft_consensus.cc:729] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d16e9369eba84097ba56d4e21ca9b2ec, State: Initialized, Role: FOLLOWER
I20260808 02:48:04.637476 19805 raft_consensus.cc:348] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:04.637852 19805 raft_consensus.cc:374] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:04.637948 19805 raft_consensus.cc:729] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cf4f4f3a1f040fb823e5dbad200dc94, State: Initialized, Role: FOLLOWER
I20260808 02:48:04.637980 19807 consensus_queue.cc:260] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:04.638475 19805 consensus_queue.cc:260] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:04.639014 19770 heartbeater.cc:503] Master 127.25.137.254:42585 was elected leader, sending a full tablet report...
I20260808 02:48:04.639595 19542 heartbeater.cc:503] Master 127.25.137.254:42585 was elected leader, sending a full tablet report...
I20260808 02:48:04.640151 19805 ts_tablet_manager.cc:1428] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260808 02:48:04.640837 19807 ts_tablet_manager.cc:1428] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
W20260808 02:48:04.734664 19771 tablet.cc:2369] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:48:04.773074 19549 tablet.cc:2369] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:48:04.824965 19300 tablet.cc:2369] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:48:04.965582 19814 raft_consensus.cc:482] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:48:04.965997 19814 raft_consensus.cc:504] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:04.967972 19814 leader_election.cc:288] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfad652e008245bfa733fa62252bf4f7 (127.25.137.193:34443), d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369)
I20260808 02:48:04.975529 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.968581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:54215 (local address 127.25.137.195:43369)
0808 02:48:04.968768 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:04.968773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:04.969536 (+ 763us) server_negotiation.cc:407] Connection header received
0808 02:48:04.969831 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:04.969837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:04.969930 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:04.970034 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:04.970601 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.971451 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.973470 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.974522 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.974804 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.974876 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:04.975128 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 02:48:04.975256 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:48:04.975553 19790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.968484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:56261 (local address 127.25.137.193:34443)
0808 02:48:04.968724 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:04.968730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:04.969945 (+ 1215us) server_negotiation.cc:407] Connection header received
0808 02:48:04.970148 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:04.970154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:04.970242 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:04.970353 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:04.970994 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.971848 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.973702 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.974513 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.974638 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.974709 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:04.975186 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 02:48:04.975326 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 02:48:04.975664 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.968420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:43369 (local address 127.25.137.194:54215)
0808 02:48:04.969327 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:04.969348 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:04.969630 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:04.970047 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:04.970052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:04.970089 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:04.970439 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.970457 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.971609 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.971615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.973316 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.973329 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.974838 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.974842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.974972 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.975001 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:48:04.975077 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:48:04.975197 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":24,"thread_start_us":778,"threads_started":1}
I20260808 02:48:04.975734 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:04.968934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34443 (local address 127.25.137.194:56261)
0808 02:48:04.969820 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:04.969838 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:04.969991 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:04.970373 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:04.970380 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:04.970404 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:04.970820 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.970840 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.971988 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.971992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.973537 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:04.973552 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:04.974831 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:04.974839 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:04.975012 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:04.975040 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 02:48:04.975130 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 02:48:04.975256 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":2,"thread_start_us":477,"threads_started":1}
I20260808 02:48:04.976671 19690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08136a859519471d9d999100f0c4c0bc" candidate_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" is_pre_election: true
I20260808 02:48:04.976701 19225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08136a859519471d9d999100f0c4c0bc" candidate_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfad652e008245bfa733fa62252bf4f7" is_pre_election: true
I20260808 02:48:04.977187 19690 raft_consensus.cc:2454] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cf4f4f3a1f040fb823e5dbad200dc94 in term 0.
I20260808 02:48:04.977188 19225 raft_consensus.cc:2454] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cf4f4f3a1f040fb823e5dbad200dc94 in term 0.
I20260808 02:48:04.977794 19336 leader_election.cc:302] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cf4f4f3a1f040fb823e5dbad200dc94, d16e9369eba84097ba56d4e21ca9b2ec; no voters:
I20260808 02:48:04.978159 19814 raft_consensus.cc:2781] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:48:04.978389 19814 raft_consensus.cc:482] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:48:04.978505 19814 raft_consensus.cc:3037] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:04.980599 19814 raft_consensus.cc:504] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:04.981216 19814 leader_election.cc:288] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [CANDIDATE]: Term 1 election: Requested vote from peers cfad652e008245bfa733fa62252bf4f7 (127.25.137.193:34443), d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369)
I20260808 02:48:04.981631 19225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08136a859519471d9d999100f0c4c0bc" candidate_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfad652e008245bfa733fa62252bf4f7"
I20260808 02:48:04.981693 19690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08136a859519471d9d999100f0c4c0bc" candidate_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d16e9369eba84097ba56d4e21ca9b2ec"
I20260808 02:48:04.981825 19225 raft_consensus.cc:3037] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:04.981890 19690 raft_consensus.cc:3037] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:04.984089 19690 raft_consensus.cc:2454] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cf4f4f3a1f040fb823e5dbad200dc94 in term 1.
I20260808 02:48:04.984091 19225 raft_consensus.cc:2454] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cf4f4f3a1f040fb823e5dbad200dc94 in term 1.
I20260808 02:48:04.984483 19341 leader_election.cc:302] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cf4f4f3a1f040fb823e5dbad200dc94, cfad652e008245bfa733fa62252bf4f7; no voters:
I20260808 02:48:04.984781 19814 raft_consensus.cc:2781] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:48:04.985301 19814 raft_consensus.cc:686] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 1 LEADER]: Becoming Leader. State: Replica: 1cf4f4f3a1f040fb823e5dbad200dc94, State: Running, Role: LEADER
I20260808 02:48:04.985831 19814 consensus_queue.cc:237] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:04.990849 18989 catalog_manager.cc:5495] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194). New cstate: current_term: 1 leader_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } health_report { overall_health: UNKNOWN } } }
I20260808 02:48:05.039026 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.033884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.25.137.194:33567)
0808 02:48:05.034048 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:05.034053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:05.034069 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:48:05.034143 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:05.034147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:05.034235 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:05.034354 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:48:05.034684 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:05.035421 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:05.036872 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:05.037380 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:05.037484 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:05.038014 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:48:05.038492 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:48:05.038610 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:05.038653 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 02:48:05.038718 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:48:05.047109 19690 raft_consensus.cc:1261] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec [term 1 FOLLOWER]: Refusing update from remote peer 1cf4f4f3a1f040fb823e5dbad200dc94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:48:05.047206 19225 raft_consensus.cc:1261] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Refusing update from remote peer 1cf4f4f3a1f040fb823e5dbad200dc94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:48:05.048323 19817 consensus_queue.cc:1035] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:48:05.048692 19814 consensus_queue.cc:1035] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:48:05.062230 19823 mvcc.cc:205] Tried to move back new op lower bound from 7316100239542837248 to 7316100239305957376. Current Snapshot: MvccSnapshot[applied={T|T < 7316100239542837248}]
I20260808 02:48:05.074935 18990 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:48:05.075229 18990 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:48:05.079032 18990 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Sending DeleteTablet for 3 replicas of tablet 08136a859519471d9d999100f0c4c0bc
I20260808 02:48:05.080125 19432 tablet_service.cc:1514] Processing DeleteTablet for tablet 08136a859519471d9d999100f0c4c0bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:05 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:36840
I20260808 02:48:05.080125 19191 tablet_service.cc:1514] Processing DeleteTablet for tablet 08136a859519471d9d999100f0c4c0bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:05 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:47658
I20260808 02:48:05.080736 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 19047
I20260808 02:48:05.080785 19650 tablet_service.cc:1514] Processing DeleteTablet for tablet 08136a859519471d9d999100f0c4c0bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:05 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:46792
I20260808 02:48:05.099126 19838 tablet_replica.cc:330] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: stopping tablet replica
I20260808 02:48:05.099789 19838 raft_consensus.cc:2229] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:48:05.100589 19838 raft_consensus.cc:2258] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:05.101284 19840 tablet_replica.cc:330] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: stopping tablet replica
I20260808 02:48:05.101742 19840 raft_consensus.cc:2229] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:48:05.101852 19841 tablet_replica.cc:330] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: stopping tablet replica
I20260808 02:48:05.102349 19840 raft_consensus.cc:2258] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:05.104251 19838 ts_tablet_manager.cc:1905] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:05.104314 19840 ts_tablet_manager.cc:1905] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:05.105614 19790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.081519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47668 (local address 127.25.137.193:34443)
0808 02:48:05.081715 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:05.081723 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:05.085850 (+ 4127us) server_negotiation.cc:407] Connection header received
0808 02:48:05.086137 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:05.086144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:05.086229 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:05.086317 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:05.088522 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:05.089758 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:05.091998 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:05.092753 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:05.092859 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:05.096993 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:05.097030 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:05.097051 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:05.097105 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:05.101556 (+ 4451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:05.102822 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:05.102832 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:05.102836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:05.103037 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:05.103653 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:05.103658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:05.103662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:05.104162 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:05.104500 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:05.104884 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 02:48:05.105152 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:48:05.106390 19288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:05.112129 19838 ts_tablet_manager.cc:1918] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:05.112391 19838 log.cc:1196] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/08136a859519471d9d999100f0c4c0bc
I20260808 02:48:05.113224 19841 raft_consensus.cc:2229] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:48:05.113827 19841 raft_consensus.cc:2258] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:05.116151 19841 ts_tablet_manager.cc:1905] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:05.116400 19838 ts_tablet_manager.cc:1939] T 08136a859519471d9d999100f0c4c0bc P 1cf4f4f3a1f040fb823e5dbad200dc94: Deleting consensus metadata
I20260808 02:48:05.118610 18972 catalog_manager.cc:4841] TS 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567): tablet 08136a859519471d9d999100f0c4c0bc (table test-table [id=01d2777020294c8aa7b650d1b3372fe3]) successfully deleted
I20260808 02:48:05.123354 19841 ts_tablet_manager.cc:1918] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:05.123534 19841 log.cc:1196] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/08136a859519471d9d999100f0c4c0bc
I20260808 02:48:05.124148 19841 ts_tablet_manager.cc:1939] T 08136a859519471d9d999100f0c4c0bc P d16e9369eba84097ba56d4e21ca9b2ec: Deleting consensus metadata
I20260808 02:48:05.126220 18967 catalog_manager.cc:4841] TS d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369): tablet 08136a859519471d9d999100f0c4c0bc (table test-table [id=01d2777020294c8aa7b650d1b3372fe3]) successfully deleted
I20260808 02:48:05.198040 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 19047
I20260808 02:48:05.227878 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 19307
W20260808 02:48:05.228080 18971 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.137.193:34443 (error 108)
I20260808 02:48:05.230366 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.229390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34443 (local address 127.0.0.1:47684)
0808 02:48:05.230003 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:05.230140 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":377,"threads_started":1}
W20260808 02:48:05.230648 18971 catalog_manager.cc:4584] TS cfad652e008245bfa733fa62252bf4f7 (127.25.137.193:34443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 08136a859519471d9d999100f0c4c0bc: Network error: Client connection negotiation failed: client connection to 127.25.137.193:34443: connect: Connection refused (error 111)
I20260808 02:48:05.242743 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.228701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36864 (local address 127.25.137.194:33567)
0808 02:48:05.228917 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:05.228924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:05.229048 (+ 124us) server_negotiation.cc:407] Connection header received
0808 02:48:05.229329 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:05.229334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:05.229432 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:05.229518 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:05.231213 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:05.232044 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:05.233474 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:05.233974 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:05.234104 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:05.236764 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:05.236778 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:05.236793 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:05.236837 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:05.239883 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:05.240553 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:05.240560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:05.240562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:05.240704 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:05.241130 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:05.241134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:05.241136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:05.241487 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:05.241846 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:05.242153 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 02:48:05.242392 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 02:48:05.243496 19536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:05.284535 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.283950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34443 (local address 127.0.0.1:47688)
0808 02:48:05.284219 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:05.284351 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 02:48:05.285811 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.285359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34443 (local address 127.0.0.1:47696)
0808 02:48:05.285541 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:05.285630 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 02:48:05.346524 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.345881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34443 (local address 127.0.0.1:47700)
0808 02:48:05.346156 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:05.346305 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 02:48:05.347702 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.347256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34443 (local address 127.0.0.1:47702)
0808 02:48:05.347414 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:05.347511 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 02:48:05.357426 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 19307
I20260808 02:48:05.381584 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 19553
I20260808 02:48:05.396340 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.382365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46796 (local address 127.25.137.195:43369)
0808 02:48:05.382560 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:05.382566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:05.382588 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:48:05.382873 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:05.382879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:05.382953 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:05.383045 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:05.384541 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:05.385358 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:05.386742 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:05.387150 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:05.387282 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:05.390303 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:05.390320 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:05.390337 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:05.390383 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:05.393249 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:05.394017 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:05.394024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:05.394027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:05.394194 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:05.394617 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:05.394622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:05.394625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:05.395044 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:05.395346 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:05.395705 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 02:48:05.395933 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:48:05.397060 19761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:05.428447 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.427898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34443 (local address 127.0.0.1:47718)
0808 02:48:05.428163 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:05.428264 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 02:48:05.429634 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.429200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34443 (local address 127.0.0.1:47720)
0808 02:48:05.429415 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:05.429489 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 02:48:05.488616 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 19553
I20260808 02:48:05.514788 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 18942
I20260808 02:48:05.516017 19037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:05.633538 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 18942
I20260808 02:48:05.633618 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:05.632998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:34443 (local address 127.0.0.1:47732)
0808 02:48:05.633375 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:05.633475 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260808 02:48:25.660060 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:42585
--webserver_interface=127.25.137.254
--webserver_port=40559
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:42585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:25.999820 19851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:26.000113 19851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:26.000234 19851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:26.007617 19851 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:48:26.007725 19851 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:48:26.007764 19851 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:48:26.007838 19851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:48:26.007913 19851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:26.008010 19851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:26.008068 19851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:48:26.008145 19851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:48:26.018110 19851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:42585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:42585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:26.019410 19851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:26.020848 19851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:26.032752 19858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:26.032833 19856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:26.032855 19860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:26.033602 19851 server_base.cc:1034] running on GCE node
I20260808 02:48:26.034375 19851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:26.035717 19851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:26.036952 19851 hybrid_clock.cc:648] HybridClock initialized: now 1786157306036874 us; error 39 us; skew 500 ppm
I20260808 02:48:26.037299 19851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:26 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157306, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:26.040336 19851 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:48:26.041855 19851 webserver.cc:460] Webserver started at http://127.25.137.254:40559/ using document root <none> and password file <none>
I20260808 02:48:26.042662 19851 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:26.042845 19851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:26.048290 19851 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:48:26.051155 19870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:26.052459 19851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:48:26.052628 19851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9"
format_stamp: "Formatted at 2026-08-08 02:47:39 on dist-test-slave-k5rr"
server_key: "341a9157efc8df857e67882e78bf274d"
server_key_iv: "be56ebeb7e14c32de55c1a9cace64450"
server_key_version: "encryptionkey@0"
I20260808 02:48:26.053640 19851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:26.071709 19851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:26.075814 19851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:26.076079 19851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:26.095072 19851 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:42585
I20260808 02:48:26.095108 19958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:42585 every 8 connection(s)
I20260808 02:48:26.096539 19851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:48:26.101778 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 19851
I20260808 02:48:26.103099 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:34443
--local_ip_for_outbound_sockets=127.25.137.193
--tserver_master_addrs=127.25.137.254:42585
--webserver_port=45979
--webserver_interface=127.25.137.193
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:48:26.104287 19960 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:48:26.108923 19960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Bootstrap starting.
I20260808 02:48:26.132539 19960 log.cc:824] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:26.150432 19960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:48:26.150974 19960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Bootstrap complete.
I20260808 02:48:26.160357 19960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:48:26.161296 19960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26e5a85104bd4a9ab1c54ff89aeb55c9, State: Initialized, Role: FOLLOWER
I20260808 02:48:26.161861 19960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:48:26.162086 19960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:48:26.162195 19960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:48:26.162362 19960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 2 FOLLOWER]: Advancing to term 3
I20260808 02:48:26.166182 19960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:48:26.166656 19960 leader_election.cc:302] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26e5a85104bd4a9ab1c54ff89aeb55c9; no voters:
I20260808 02:48:26.167439 19960 leader_election.cc:288] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 02:48:26.167585 19967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 02:48:26.168758 19967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [term 3 LEADER]: Becoming Leader. State: Replica: 26e5a85104bd4a9ab1c54ff89aeb55c9, State: Running, Role: LEADER
I20260808 02:48:26.169348 19967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } }
I20260808 02:48:26.169672 19960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:48:26.171813 19969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } } }
I20260808 02:48:26.172206 19969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:26.172469 19971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26e5a85104bd4a9ab1c54ff89aeb55c9. Latest consensus state: current_term: 3 leader_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e5a85104bd4a9ab1c54ff89aeb55c9" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 42585 } } }
I20260808 02:48:26.173131 19971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:26.177762 19976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:48:26.187282 19976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:48:26.188832 19976 catalog_manager.cc:1259] Loaded cluster ID: 821950171b7a45c3914a5f60aa5628bd
I20260808 02:48:26.188979 19976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:48:26.192935 19976 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:48:26.194634 19976 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Loaded TSK: 0
I20260808 02:48:26.194852 19976 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 26e5a85104bd4a9ab1c54ff89aeb55c9: Loaded TSK: 1
F20260808 02:48:26.195989 19976 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f244d8d29ad google::LogMessage::Fail() at ??:0
I20260808 02:48:26.199877 19991 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f244d8d588a google::LogMessage::SendToLog() at ??:0
@ 0x7f244d8d24de google::LogMessage::Flush() at ??:0
@ 0x7f244d8d3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f24516367d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f245162573e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f24633011c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f2463300783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f246338c2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f246338c0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f24633d65e9 std::function<>::operator()() at ??:0
@ 0x7f2463304cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f2463302c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f246338a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f246338a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f245051e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f245052191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f2450521799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f24504fceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f24537d66db start_thread at ??:0
@ 0x7f244c76c61f clone at ??:0
W20260808 02:48:26.496862 19964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:26.497145 19964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:26.497282 19964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:26.504403 19964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:26.504510 19964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:26.504663 19964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:48:26.514635 19964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:34443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=45979
--tserver_master_addrs=127.25.137.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:26.515923 19964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:26.517290 19964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:26.530119 20030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:26.530206 20035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:26.530160 20031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:26.531229 19964 server_base.cc:1034] running on GCE node
I20260808 02:48:26.531845 19964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:26.532737 19964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:26.533954 19964 hybrid_clock.cc:648] HybridClock initialized: now 1786157306533891 us; error 55 us; skew 500 ppm
I20260808 02:48:26.534266 19964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:26 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157306, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:26.537458 19964 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:48:26.538882 19964 webserver.cc:460] Webserver started at http://127.25.137.193:45979/ using document root <none> and password file <none>
I20260808 02:48:26.539548 19964 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:26.539685 19964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:26.545038 19964 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:48:26.547950 20046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:26.549407 19964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 02:48:26.549582 19964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "cfad652e008245bfa733fa62252bf4f7"
format_stamp: "Formatted at 2026-08-08 02:47:40 on dist-test-slave-k5rr"
server_key: "4176e593ac8029ebe8f80c34b4cf518e"
server_key_iv: "ccbb8254ef44316f2dbcffcf001d9402"
server_key_version: "encryptionkey@0"
I20260808 02:48:26.550631 19964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:26.581562 19964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:26.585969 19964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:26.586230 19964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:26.587263 19964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:26.589166 20061 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260808 02:48:26.594887 19964 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:48:26.595006 19964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 02:48:26.595124 19964 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:48:26.598299 19964 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:48:26.598377 19964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.002s
I20260808 02:48:26.598723 20061 tablet_bootstrap.cc:492] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Bootstrap starting.
I20260808 02:48:26.607219 20061 log.cc:824] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:26.622953 20061 tablet_bootstrap.cc:492] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:48:26.623744 20061 tablet_bootstrap.cc:492] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Bootstrap complete.
I20260808 02:48:26.624279 20061 ts_tablet_manager.cc:1397] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Time spent bootstrapping tablet: real 0.026s user 0.020s sys 0.004s
I20260808 02:48:26.633926 20061 raft_consensus.cc:348] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:26.635167 20061 raft_consensus.cc:729] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfad652e008245bfa733fa62252bf4f7, State: Initialized, Role: FOLLOWER
I20260808 02:48:26.635680 19964 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:34443
I20260808 02:48:26.635735 20241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:34443 every 8 connection(s)
I20260808 02:48:26.635946 20061 consensus_queue.cc:260] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:26.637033 19964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:48:26.637653 20061 ts_tablet_manager.cc:1428] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260808 02:48:26.642457 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:26.639912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:36489)
0808 02:48:26.640863 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:26.641223 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":388,"threads_started":1}
W20260808 02:48:26.643193 20243 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:42585 (0 consecutive failures): Network error: Failed to ping master at 127.25.137.254:42585: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
I20260808 02:48:26.644140 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:26.643845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:49135)
0808 02:48:26.643979 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:26.644045 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 02:48:26.644944 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:26.644619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:56289)
0808 02:48:26.644753 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:26.644830 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 02:48:26.645195 20243 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:48:26.646574 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 19964
I20260808 02:48:26.647192 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:33567
--local_ip_for_outbound_sockets=127.25.137.194
--tserver_master_addrs=127.25.137.254:42585
--webserver_port=39025
--webserver_interface=127.25.137.194
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:26.979358 20246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:26.979640 20246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:26.979832 20246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:26.986917 20246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:26.987035 20246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:26.987193 20246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:48:26.996750 20246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:33567
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=39025
--tserver_master_addrs=127.25.137.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:26.998021 20246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:26.999372 20246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:27.012187 20253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:27.012233 20254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:27.012257 20257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:27.013036 20246 server_base.cc:1034] running on GCE node
I20260808 02:48:27.013707 20246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:27.014609 20246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:27.015798 20246 hybrid_clock.cc:648] HybridClock initialized: now 1786157307015725 us; error 69 us; skew 500 ppm
I20260808 02:48:27.016105 20246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:27 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157307, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:27.019140 20246 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:48:27.020504 20246 webserver.cc:460] Webserver started at http://127.25.137.194:39025/ using document root <none> and password file <none>
I20260808 02:48:27.021178 20246 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:27.021286 20246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:27.026605 20246 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 02:48:27.029248 20266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:27.030520 20246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:48:27.030665 20246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "1cf4f4f3a1f040fb823e5dbad200dc94"
format_stamp: "Formatted at 2026-08-08 02:47:40 on dist-test-slave-k5rr"
server_key: "6c25a695052bc74635c4306983987233"
server_key_iv: "07390db24b0797a569c764f2a6c9f70b"
server_key_version: "encryptionkey@0"
I20260808 02:48:27.031654 20246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:27.058877 20246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:27.062649 20246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:27.062902 20246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:27.063884 20246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:27.065317 20246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:27.065423 20246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:27.065522 20246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:27.065590 20246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:27.095934 20246 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:33567
I20260808 02:48:27.096005 20445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:33567 every 8 connection(s)
I20260808 02:48:27.097388 20246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:48:27.102159 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:27.100302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:46195)
0808 02:48:27.101027 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:27.101289 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":260,"threads_started":1}
W20260808 02:48:27.102834 20446 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:42585 (0 consecutive failures): Network error: Failed to ping master at 127.25.137.254:42585: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
I20260808 02:48:27.103672 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:27.103325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:36137)
0808 02:48:27.103493 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:27.103566 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 02:48:27.104480 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:27.104161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:59457)
0808 02:48:27.104267 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:27.104372 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 02:48:27.104710 20446 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:48:27.107280 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 20246
I20260808 02:48:27.107949 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:43369
--local_ip_for_outbound_sockets=127.25.137.195
--tserver_master_addrs=127.25.137.254:42585
--webserver_port=35039
--webserver_interface=127.25.137.195
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:27.442483 20450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:27.442804 20450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:27.442962 20450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:27.450747 20450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:27.450898 20450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:27.451084 20450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:48:27.461233 20450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:43369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=35039
--tserver_master_addrs=127.25.137.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:27.462595 20450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:27.463958 20450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:27.477065 20459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:27.477087 20466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:27.477558 20463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:27.478272 20450 server_base.cc:1034] running on GCE node
I20260808 02:48:27.479002 20450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:27.480189 20450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:27.481504 20450 hybrid_clock.cc:648] HybridClock initialized: now 1786157307481404 us; error 54 us; skew 500 ppm
I20260808 02:48:27.481907 20450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:27 dist-test-slave-k5rr krb5kdc[17992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157307, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:27.485184 20450 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:48:27.486697 20450 webserver.cc:460] Webserver started at http://127.25.137.195:35039/ using document root <none> and password file <none>
I20260808 02:48:27.487396 20450 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:27.487512 20450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:27.493182 20450 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:48:27.495862 20476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:27.497232 20450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:48:27.497390 20450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "d16e9369eba84097ba56d4e21ca9b2ec"
format_stamp: "Formatted at 2026-08-08 02:47:41 on dist-test-slave-k5rr"
server_key: "3b2229831af9ea285303d7fc73680907"
server_key_iv: "00d55b46b18dec95eb4d85b90b1b4fc7"
server_key_version: "encryptionkey@0"
I20260808 02:48:27.498381 20450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:27.525825 20450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:27.530375 20450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:27.530674 20450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:27.531715 20450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:27.533207 20450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:27.533311 20450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:27.533432 20450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:27.533504 20450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:27.573225 20450 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:43369
I20260808 02:48:27.573282 20658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:43369 every 8 connection(s)
I20260808 02:48:27.574651 20450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:48:27.579377 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 20450
I20260808 02:48:27.579511 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:27.577614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:59035)
0808 02:48:27.578367 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:27.578612 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":284,"threads_started":1}
W20260808 02:48:27.580220 20659 heartbeater.cc:644] Failed to heartbeat to 127.25.137.254:42585 (0 consecutive failures): Network error: Failed to ping master at 127.25.137.254:42585: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
I20260808 02:48:27.581161 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:27.580751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:50579)
0808 02:48:27.580926 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:27.580996 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 02:48:27.581929 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:27.581619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:44645)
0808 02:48:27.581727 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:27.581799 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 02:48:27.582134 20659 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:48:27.646523 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:27.645740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:34883)
0808 02:48:27.646208 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:27.646297 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":212,"threads_started":1}
I20260808 02:48:28.106112 20666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:28.105315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:33611)
0808 02:48:28.105777 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:28.105874 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":217,"threads_started":1}
I20260808 02:48:28.326490 20667 raft_consensus.cc:482] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:48:28.326766 20667 raft_consensus.cc:504] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:28.328532 20667 leader_election.cc:288] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567), d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369)
Aug 08 02:48:28 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157306, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
Aug 08 02:48:28 dist-test-slave-k5rr krb5kdc[17992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157306, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:48:28.361464 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:28.329182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:41483 (local address 127.25.137.195:43369)
0808 02:48:28.340417 (+ 11235us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:28.340438 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:28.341496 (+ 1058us) server_negotiation.cc:407] Connection header received
0808 02:48:28.343383 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:28.343392 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:28.343749 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:28.344044 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:28.346149 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:28.348018 (+ 1869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:28.349834 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:28.350458 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:28.350638 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:28.354807 (+ 4169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:28.354814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:28.354825 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:28.354860 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:28.357931 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:28.358624 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:28.358629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:28.358635 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:28.358783 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:28.359210 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:28.359214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:28.359216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:28.359713 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:28.359905 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:28.360666 (+ 761us) server_negotiation.cc:299] Negotiation successful
0808 02:48:28.361036 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11006,"thread_start_us":10898,"threads_started":1}
I20260808 02:48:28.361479 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:28.329154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:39007 (local address 127.25.137.194:33567)
0808 02:48:28.340882 (+ 11728us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:28.340900 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:28.341363 (+ 463us) server_negotiation.cc:407] Connection header received
0808 02:48:28.343517 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:28.343525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:28.343836 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:28.344121 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:28.346182 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:28.348030 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:28.349828 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:28.350460 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:28.350628 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:28.354542 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:28.354551 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:28.354569 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:28.354614 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:28.357806 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:28.358586 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:28.358594 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:28.358602 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:28.358756 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:28.359216 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:28.359221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:28.359225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:28.359710 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:28.359901 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:28.360691 (+ 790us) server_negotiation.cc:299] Negotiation successful
0808 02:48:28.361096 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11502,"thread_start_us":11294,"threads_started":1}
I20260808 02:48:28.361488 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:28.328715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:33567 (local address 127.25.137.193:39007)
0808 02:48:28.341173 (+ 12458us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:28.341221 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:28.341775 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:28.344191 (+ 2416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:28.344201 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:28.345173 (+ 972us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:28.345957 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:28.345978 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:28.348231 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:28.348235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:28.349692 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:28.349705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:28.350661 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:28.350668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:28.350979 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:28.351671 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:48:28.351697 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:28.354246 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:28.357946 (+ 3700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:28.357954 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:28.357968 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:28.358443 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:28.358875 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:28.358880 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:28.358884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:28.359098 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:28.359849 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:28.359857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:28.360185 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 02:48:28.360433 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 02:48:28.360975 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12331,"mutex_wait_us":161,"spinlock_wait_cycles":61184,"thread_start_us":12185,"threads_started":1}
I20260808 02:48:28.362228 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:28.328879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:43369 (local address 127.25.137.193:41483)
0808 02:48:28.341385 (+ 12506us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:28.341401 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:28.341764 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:28.344194 (+ 2430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:28.344202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:28.345165 (+ 963us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:28.345962 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:28.345986 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:28.348242 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:28.348246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:28.349687 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:28.349700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:28.350710 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:28.350718 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:28.350975 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:28.351735 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:48:28.351752 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:28.354556 (+ 2804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:28.358052 (+ 3496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:28.358057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:28.358061 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:28.358497 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:28.358902 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:28.358908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:28.358912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:28.359098 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:28.359848 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:28.359855 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:28.360187 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 02:48:28.360434 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 02:48:28.361152 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12381,"mutex_wait_us":345,"spinlock_wait_cycles":45696,"thread_start_us":12225,"threads_started":1}
W20260808 02:48:28.364897 20048 leader_election.cc:341] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369): Not found: Tablet not found: 08136a859519471d9d999100f0c4c0bc
W20260808 02:48:28.365202 20056 leader_election.cc:341] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567): Not found: Tablet not found: 08136a859519471d9d999100f0c4c0bc
I20260808 02:48:28.365351 20056 leader_election.cc:302] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfad652e008245bfa733fa62252bf4f7; no voters: 1cf4f4f3a1f040fb823e5dbad200dc94, d16e9369eba84097ba56d4e21ca9b2ec
I20260808 02:48:28.365725 20667 raft_consensus.cc:2726] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:48:28.583554 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:28.582646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:56829)
0808 02:48:28.583126 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:28.583278 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":240,"threads_started":1}
I20260808 02:48:28.647858 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:28.647395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:57831)
0808 02:48:28.647582 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:28.647678 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 02:48:29.107791 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:29.107047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:41559)
0808 02:48:29.107537 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:29.107629 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":264,"threads_started":1}
I20260808 02:48:29.585208 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:29.584455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:49941)
0808 02:48:29.584935 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:29.585049 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":252,"threads_started":1}
I20260808 02:48:29.649521 20680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:29.648769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:48787)
0808 02:48:29.649252 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:29.649342 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":251,"threads_started":1}
I20260808 02:48:30.109632 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:30.108804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:33029)
0808 02:48:30.109357 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:30.109444 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":297,"threads_started":1}
I20260808 02:48:30.453169 20683 raft_consensus.cc:482] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:48:30.453369 20683 raft_consensus.cc:504] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:30.454092 20683 leader_election.cc:288] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567), d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369)
W20260808 02:48:30.455137 20048 leader_election.cc:341] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369): Not found: Tablet not found: 08136a859519471d9d999100f0c4c0bc
W20260808 02:48:30.455350 20056 leader_election.cc:341] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567): Not found: Tablet not found: 08136a859519471d9d999100f0c4c0bc
I20260808 02:48:30.455438 20056 leader_election.cc:302] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfad652e008245bfa733fa62252bf4f7; no voters: 1cf4f4f3a1f040fb823e5dbad200dc94, d16e9369eba84097ba56d4e21ca9b2ec
I20260808 02:48:30.455646 20683 raft_consensus.cc:2726] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:48:30.586839 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:30.586103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:37343)
0808 02:48:30.586553 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:30.586639 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":229,"threads_started":1}
I20260808 02:48:30.651190 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:30.650375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:54107)
0808 02:48:30.650854 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:30.650983 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":232,"threads_started":1}
I20260808 02:48:31.111271 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:31.110557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:52011)
0808 02:48:31.111008 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:31.111089 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":204,"threads_started":1}
I20260808 02:48:31.588469 20688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:31.587670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:41599)
0808 02:48:31.588157 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:31.588255 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":253,"threads_started":1}
I20260808 02:48:31.653074 20689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:31.652230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:50629)
0808 02:48:31.652770 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:31.652859 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":234,"threads_started":1}
I20260808 02:48:32.112977 20691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:32.112201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:44003)
0808 02:48:32.112668 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:32.112754 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":242,"threads_started":1}
I20260808 02:48:32.590221 20693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:32.589407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:33581)
0808 02:48:32.589972 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:32.590057 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":277,"threads_started":1}
I20260808 02:48:32.654847 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:32.654054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:50611)
0808 02:48:32.654536 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:32.654668 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":252,"threads_started":1}
I20260808 02:48:33.114655 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:33.113887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:57071)
0808 02:48:33.114374 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:33.114458 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":248,"threads_started":1}
I20260808 02:48:33.211200 20696 raft_consensus.cc:482] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:48:33.211404 20696 raft_consensus.cc:504] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:33.212128 20696 leader_election.cc:288] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567), d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369)
W20260808 02:48:33.213141 20048 leader_election.cc:341] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369): Not found: Tablet not found: 08136a859519471d9d999100f0c4c0bc
W20260808 02:48:33.213358 20056 leader_election.cc:341] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567): Not found: Tablet not found: 08136a859519471d9d999100f0c4c0bc
I20260808 02:48:33.213492 20056 leader_election.cc:302] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfad652e008245bfa733fa62252bf4f7; no voters: 1cf4f4f3a1f040fb823e5dbad200dc94, d16e9369eba84097ba56d4e21ca9b2ec
I20260808 02:48:33.213719 20696 raft_consensus.cc:2726] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:48:33.591907 20697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:33.591157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:46515)
0808 02:48:33.591641 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:33.591723 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":255,"threads_started":1}
I20260808 02:48:33.656462 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:33.655611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:52797)
0808 02:48:33.656108 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:33.656223 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":258,"threads_started":1}
I20260808 02:48:34.116532 20699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:34.115670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:57453)
0808 02:48:34.116229 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:34.116340 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":242,"threads_started":1}
I20260808 02:48:34.593662 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:34.592771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:48639)
0808 02:48:34.593365 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:34.593456 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":276,"threads_started":1}
I20260808 02:48:34.658077 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:34.657354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:37869)
0808 02:48:34.657824 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:34.657921 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":229,"threads_started":1}
I20260808 02:48:34.974836 20703 raft_consensus.cc:482] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:48:34.975044 20703 raft_consensus.cc:504] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:34.975718 20703 leader_election.cc:288] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567), d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369)
W20260808 02:48:34.976861 20048 leader_election.cc:341] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369): Not found: Tablet not found: 08136a859519471d9d999100f0c4c0bc
W20260808 02:48:34.977075 20056 leader_election.cc:341] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567): Not found: Tablet not found: 08136a859519471d9d999100f0c4c0bc
I20260808 02:48:34.977190 20056 leader_election.cc:302] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfad652e008245bfa733fa62252bf4f7; no voters: 1cf4f4f3a1f040fb823e5dbad200dc94, d16e9369eba84097ba56d4e21ca9b2ec
I20260808 02:48:34.977451 20703 raft_consensus.cc:2726] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:48:35.118294 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:35.117563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:58803)
0808 02:48:35.118010 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:35.118107 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":244,"threads_started":1}
I20260808 02:48:35.595300 20706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:35.594556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:44935)
0808 02:48:35.595022 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:35.595103 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":260,"threads_started":1}
I20260808 02:48:35.659647 20707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:35.658924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:49887)
0808 02:48:35.659382 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:35.659469 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":240,"threads_started":1}
I20260808 02:48:36.120252 20709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:36.119359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:58387)
0808 02:48:36.119950 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:36.120054 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":280,"threads_started":1}
I20260808 02:48:36.596982 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:36.596208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:35233)
0808 02:48:36.596728 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:36.596826 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":272,"threads_started":1}
I20260808 02:48:36.661329 20711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:36.660553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:58925)
0808 02:48:36.661048 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:36.661148 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":244,"threads_started":1}
I20260808 02:48:37.122342 20713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:37.121440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:37781)
0808 02:48:37.122061 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:37.122149 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":350,"threads_started":1}
I20260808 02:48:37.598686 20716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:37.597945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:52735)
0808 02:48:37.598415 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:37.598499 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":236,"threads_started":1}
I20260808 02:48:37.663075 20717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:37.662322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:37769)
0808 02:48:37.662766 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:37.662893 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":225,"threads_started":1}
I20260808 02:48:38.124096 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:38.123295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:47217)
0808 02:48:38.123793 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:38.123882 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":235,"threads_started":1}
I20260808 02:48:38.600314 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:38.599529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:50937)
0808 02:48:38.600013 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:38.600126 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":252,"threads_started":1}
I20260808 02:48:38.664633 20722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:38.663912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:50675)
0808 02:48:38.664368 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:38.664465 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":231,"threads_started":1}
I20260808 02:48:39.125819 20725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:39.125000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:40817)
0808 02:48:39.125529 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:39.125614 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":257,"threads_started":1}
I20260808 02:48:39.602352 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:39.601465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:53581)
0808 02:48:39.602022 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:39.602142 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":278,"threads_started":1}
I20260808 02:48:39.666352 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:39.665534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:50581)
0808 02:48:39.666021 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:39.666123 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":240,"threads_started":1}
I20260808 02:48:40.127549 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:40.126803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:59919)
0808 02:48:40.127291 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:40.127385 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":250,"threads_started":1}
I20260808 02:48:40.604008 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:40.603239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:60389)
0808 02:48:40.603675 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:40.603787 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":220,"threads_started":1}
I20260808 02:48:40.667977 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:40.667216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:46169)
0808 02:48:40.667676 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:40.667754 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":239,"threads_started":1}
I20260808 02:48:41.129419 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:41.128545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:45255)
0808 02:48:41.129102 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:41.129196 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":270,"threads_started":1}
I20260808 02:48:41.334332 20739 raft_consensus.cc:482] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:48:41.334558 20739 raft_consensus.cc:504] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfad652e008245bfa733fa62252bf4f7" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 34443 } } peers { permanent_uuid: "1cf4f4f3a1f040fb823e5dbad200dc94" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 33567 } } peers { permanent_uuid: "d16e9369eba84097ba56d4e21ca9b2ec" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 43369 } }
I20260808 02:48:41.335268 20739 leader_election.cc:288] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567), d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369)
W20260808 02:48:41.336269 20056 leader_election.cc:341] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cf4f4f3a1f040fb823e5dbad200dc94 (127.25.137.194:33567): Not found: Tablet not found: 08136a859519471d9d999100f0c4c0bc
W20260808 02:48:41.336459 20048 leader_election.cc:341] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d16e9369eba84097ba56d4e21ca9b2ec (127.25.137.195:43369): Not found: Tablet not found: 08136a859519471d9d999100f0c4c0bc
I20260808 02:48:41.336563 20048 leader_election.cc:302] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfad652e008245bfa733fa62252bf4f7; no voters: 1cf4f4f3a1f040fb823e5dbad200dc94, d16e9369eba84097ba56d4e21ca9b2ec
I20260808 02:48:41.336809 20739 raft_consensus.cc:2726] T 08136a859519471d9d999100f0c4c0bc P cfad652e008245bfa733fa62252bf4f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:48:41.605731 20740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:41.604925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:33065)
0808 02:48:41.605430 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:41.605528 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":314,"threads_started":1}
I20260808 02:48:41.670091 20741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:41.669092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:45067)
0808 02:48:41.669666 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:41.669849 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":272,"threads_started":1}
I20260808 02:48:42.131506 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:42.130645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.194:43133)
0808 02:48:42.131190 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:42.131311 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":275,"threads_started":1}
I20260808 02:48:42.589845 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 19964
I20260808 02:48:42.609696 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:42.606830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.195:44225)
0808 02:48:42.609434 (+ 2604us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:42.609535 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2499,"thread_start_us":2372,"threads_started":1}
I20260808 02:48:42.609743 20743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:42.590934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56186 (local address 127.25.137.193:34443)
0808 02:48:42.593188 (+ 2254us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:42.593197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:42.593226 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:48:42.593295 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:42.593302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:42.593516 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:42.593649 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:42.596920 (+ 3271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:42.598169 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:42.599732 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:42.600501 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:42.600608 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:42.603450 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:42.603463 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:42.603479 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:42.603518 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:42.606469 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:42.607264 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:42.607274 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:42.607277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:42.607471 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:42.607820 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:42.607825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:42.607828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:42.608335 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:42.608673 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:42.608942 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 02:48:42.609441 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2090,"thread_start_us":1952,"threads_started":1}
I20260808 02:48:42.611241 20230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:42.701422 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:42.700269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:42585 (local address 127.25.137.193:57175)
0808 02:48:42.700856 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:42.701080 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.254:42585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":291,"threads_started":1}
I20260808 02:48:42.701488 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 19964
I20260808 02:48:42.730054 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 20246
I20260808 02:48:42.742870 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:42.730864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.25.137.194:33567)
0808 02:48:42.731351 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:42.731356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:42.731373 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:48:42.731430 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:42.731435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:42.731510 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:42.731601 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:42.733043 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:42.733859 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:42.735092 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:42.735596 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:42.735683 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:42.738203 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:42.738208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:42.738211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:42.738242 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:42.740658 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:42.741310 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:42.741314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:42.741317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:42.741437 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:42.741725 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:42.741728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:42.741730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:42.741989 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:42.742189 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:42.742417 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 02:48:42.742584 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":261,"threads_started":1}
I20260808 02:48:42.743547 20438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:42.827239 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 20246
I20260808 02:48:42.850735 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 20450
I20260808 02:48:42.864778 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:42.851673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.25.137.195:43369)
0808 02:48:42.852347 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:42.852352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:42.852369 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:48:42.852426 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:42.852430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:42.852489 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:42.852585 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:42.854170 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:42.854962 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:42.856267 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:42.856723 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:42.856835 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:42.859439 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:42.859444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:42.859448 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:42.859481 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:42.862191 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:42.862866 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:42.862871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:42.862873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:42.863025 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:42.863379 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:42.863383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:42.863385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:42.863636 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:42.863898 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:42.864177 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 02:48:42.864405 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":369,"threads_started":1}
I20260808 02:48:42.865598 20650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:42.950115 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 20450
2026-08-08T02:48:42Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64990 ms)
[----------] 30 tests from SecurityITest (303612 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:48:43 dist-test-slave-k5rr krb5kdc[20759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:48:43 dist-test-slave-k5rr krb5kdc[20759](info): set up 2 sockets
Aug 08 02:48:43 dist-test-slave-k5rr krb5kdc[20759](info): commencing operation
krb5kdc: starting...
W20260808 02:48:44.189579 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:48:44 dist-test-slave-k5rr krb5kdc[20759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:48:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:48:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:48:44.504264 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:41319
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:33965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:41319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:44.844002 20780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:44.844266 20780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:44.844384 20780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:44.851437 20780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:48:44.851563 20780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:44.851621 20780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:44.851701 20780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:48:44.851756 20780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:48:44.861526 20780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:41319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:41319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:44.862753 20780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:44.864081 20780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:44.875857 20789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:44.875897 20791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:44.876127 20787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:44.876895 20780 server_base.cc:1034] running on GCE node
I20260808 02:48:44.877705 20780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:44.879122 20780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:44.880338 20780 hybrid_clock.cc:648] HybridClock initialized: now 1786157324880289 us; error 30 us; skew 500 ppm
I20260808 02:48:44.880671 20780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:44 dist-test-slave-k5rr krb5kdc[20759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157324, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:44.886857 20780 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:48:44.888509 20780 webserver.cc:460] Webserver started at http://127.25.137.254:46367/ using document root <none> and password file <none>
I20260808 02:48:44.889276 20780 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:44.889410 20780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:44.889729 20780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:44.892907 20780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "4d27e43abc6540f2b58540d3c2df695e"
format_stamp: "Formatted at 2026-08-08 02:48:44 on dist-test-slave-k5rr"
server_key: "6b30ee3a6c61a6f8377cdff6615d8d9c"
server_key_iv: "6d465e71521e413bccdb291f1ca85a42"
server_key_version: "encryptionkey@0"
I20260808 02:48:44.893759 20780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "4d27e43abc6540f2b58540d3c2df695e"
format_stamp: "Formatted at 2026-08-08 02:48:44 on dist-test-slave-k5rr"
server_key: "6b30ee3a6c61a6f8377cdff6615d8d9c"
server_key_iv: "6d465e71521e413bccdb291f1ca85a42"
server_key_version: "encryptionkey@0"
I20260808 02:48:44.899101 20780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 02:48:44.903658 20803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:44.905092 20780 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:48:44.905249 20780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "4d27e43abc6540f2b58540d3c2df695e"
format_stamp: "Formatted at 2026-08-08 02:48:44 on dist-test-slave-k5rr"
server_key: "6b30ee3a6c61a6f8377cdff6615d8d9c"
server_key_iv: "6d465e71521e413bccdb291f1ca85a42"
server_key_version: "encryptionkey@0"
I20260808 02:48:44.905419 20780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:44.918934 20780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:44.923002 20780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:44.923254 20780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:44.942193 20780 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:41319
I20260808 02:48:44.942194 20882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:41319 every 8 connection(s)
I20260808 02:48:44.943692 20780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:48:44.944958 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 20780
I20260808 02:48:44.945207 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:48:44.946084 26151 external_mini_cluster.cc:1428] Setting key 411ac410464b8cd21d56f5dc4b77a7b6
I20260808 02:48:44.948566 20883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 02:48:44 dist-test-slave-k5rr krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:48:44.962127 20883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e: Bootstrap starting.
I20260808 02:48:44.965690 20883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:44.966902 20883 log.cc:824] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:44.970112 20883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e: No bootstrap required, opened a new log
I20260808 02:48:44.970530 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:44.948363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53800 (local address 127.25.137.254:41319)
0808 02:48:44.949252 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:44.949265 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:44.949310 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:48:44.951084 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:44.951106 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:44.951525 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:44.951886 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:44.953737 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:44.955151 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:44.957040 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:44.957830 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:44.957970 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:44.962974 (+ 5004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:44.962983 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:44.962998 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:44.963037 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:44.966145 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:44.966999 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:44.967005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:44.967011 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:44.967208 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:44.967671 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:44.967676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:44.967678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:44.968405 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:44.968611 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:44.969144 (+ 533us) server_negotiation.cc:299] Negotiation successful
0808 02:48:44.969479 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":401,"threads_started":1}
I20260808 02:48:44.977515 20883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27e43abc6540f2b58540d3c2df695e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41319 } }
I20260808 02:48:44.977874 20883 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:44.977965 20883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d27e43abc6540f2b58540d3c2df695e, State: Initialized, Role: FOLLOWER
I20260808 02:48:44.978519 20883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27e43abc6540f2b58540d3c2df695e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41319 } }
I20260808 02:48:44.978740 20883 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:48:44.978883 20883 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:48:44.979040 20883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:44.981365 20883 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27e43abc6540f2b58540d3c2df695e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41319 } }
I20260808 02:48:44.981889 20883 leader_election.cc:302] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d27e43abc6540f2b58540d3c2df695e; no voters:
I20260808 02:48:44.982455 20883 leader_election.cc:288] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:48:44.982592 20891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:48:44.983362 20891 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [term 1 LEADER]: Becoming Leader. State: Replica: 4d27e43abc6540f2b58540d3c2df695e, State: Running, Role: LEADER
I20260808 02:48:44.983891 20891 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27e43abc6540f2b58540d3c2df695e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41319 } }
I20260808 02:48:44.984625 20883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:48:44.986596 20895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d27e43abc6540f2b58540d3c2df695e. Latest consensus state: current_term: 1 leader_uuid: "4d27e43abc6540f2b58540d3c2df695e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27e43abc6540f2b58540d3c2df695e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41319 } } }
I20260808 02:48:44.987010 20895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:44.987491 20894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d27e43abc6540f2b58540d3c2df695e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27e43abc6540f2b58540d3c2df695e" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 41319 } } }
I20260808 02:48:44.987802 20894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:44.988060 20901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:48:44.994596 20901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:48:45.004401 20901 catalog_manager.cc:1347] Generated new cluster ID: bf20921f9cf842f9af2d194436370f94
I20260808 02:48:45.004511 20901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:48:45.022848 20901 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:48:45.024147 20901 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:48:45.038316 20901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e: Generated new TSK 0
I20260808 02:48:45.039147 20901 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:48:45.112917 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41319
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:33965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:45.451103 20924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:45.451380 20924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:45.451527 20924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:45.458961 20924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:45.459097 20924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:45.459270 20924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:48:45.469242 20924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:45.470652 20924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:45.472169 20924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:45.486507 20932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:45.486701 20938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:45.486550 20931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:45.487509 20924 server_base.cc:1034] running on GCE node
I20260808 02:48:45.488076 20924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:45.489151 20924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:45.490350 20924 hybrid_clock.cc:648] HybridClock initialized: now 1786157325490294 us; error 56 us; skew 500 ppm
I20260808 02:48:45.490640 20924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:45 dist-test-slave-k5rr krb5kdc[20759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157325, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:45.493989 20924 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:48:45.495316 20924 webserver.cc:460] Webserver started at http://127.25.137.193:37711/ using document root <none> and password file <none>
I20260808 02:48:45.495954 20924 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:45.496058 20924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:45.496321 20924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:45.499215 20924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "bd0cc1df317c4f118b250e8c8f7462f1"
format_stamp: "Formatted at 2026-08-08 02:48:45 on dist-test-slave-k5rr"
server_key: "d4944fb73df0876eee56dd8f51e78104"
server_key_iv: "35705769df432134f25c246a4878d694"
server_key_version: "encryptionkey@0"
I20260808 02:48:45.499929 20924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "bd0cc1df317c4f118b250e8c8f7462f1"
format_stamp: "Formatted at 2026-08-08 02:48:45 on dist-test-slave-k5rr"
server_key: "d4944fb73df0876eee56dd8f51e78104"
server_key_iv: "35705769df432134f25c246a4878d694"
server_key_version: "encryptionkey@0"
I20260808 02:48:45.505122 20924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:48:45.508458 20946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:45.509776 20924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:48:45.509958 20924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "bd0cc1df317c4f118b250e8c8f7462f1"
format_stamp: "Formatted at 2026-08-08 02:48:45 on dist-test-slave-k5rr"
server_key: "d4944fb73df0876eee56dd8f51e78104"
server_key_iv: "35705769df432134f25c246a4878d694"
server_key_version: "encryptionkey@0"
I20260808 02:48:45.510161 20924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:45.530987 20924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:45.535215 20924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:45.535516 20924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:45.536458 20924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:45.537945 20924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:45.538034 20924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:45.538154 20924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:45.538215 20924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:45.568598 20924 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:45863
I20260808 02:48:45.568646 21125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:45863 every 8 connection(s)
I20260808 02:48:45.569876 20924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:48:45.575347 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 20924
I20260808 02:48:45.575536 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:48:45.576157 26151 external_mini_cluster.cc:1428] Setting key febe659d17daad44c47cf7a57bcdab2e
Aug 08 02:48:45 dist-test-slave-k5rr krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157325, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:48:45.590826 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:45.572556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:43277 (local address 127.25.137.254:41319)
0808 02:48:45.573034 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:45.573038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:45.573966 (+ 928us) server_negotiation.cc:407] Connection header received
0808 02:48:45.575875 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:45.575881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:45.575966 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:45.576062 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:45.577916 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:45.578949 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:45.580603 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:45.581179 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:45.581314 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:45.584945 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:45.584951 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:45.584954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:45.584986 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:45.587814 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:45.588512 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:45.588516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:45.588519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:45.588641 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:45.589068 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:45.589072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:45.589075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:45.589381 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:45.589517 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:45.590226 (+ 709us) server_negotiation.cc:299] Negotiation successful
0808 02:48:45.590448 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":232,"threads_started":1}
I20260808 02:48:45.591840 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:45.573109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41319 (local address 127.25.137.193:43277)
0808 02:48:45.573817 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:45.573854 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:45.575596 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:45.576332 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:45.576354 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:45.576942 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:45.577707 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:45.577732 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:45.579143 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:45.579147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:45.580446 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:45.580459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:45.581412 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:45.581417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:45.581635 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:45.582340 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:48:45.582368 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:45.584724 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:45.587945 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:45.587953 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:45.587970 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:45.588400 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:45.588746 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:45.588751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:45.588755 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:45.588934 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:45.589510 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:45.589516 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:45.589841 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 02:48:45.590085 (+ 244us) client_negotiation.cc:240] Negotiation successful
0808 02:48:45.590609 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":272,"threads_started":1}
I20260808 02:48:45.593367 21126 heartbeater.cc:347] Connected to a master server at 127.25.137.254:41319
I20260808 02:48:45.593657 21126 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:45.594354 21126 heartbeater.cc:511] Master 127.25.137.254:41319 requested a full tablet report, sending...
I20260808 02:48:45.596529 20827 ts_manager.cc:194] Registered new tserver with Master: bd0cc1df317c4f118b250e8c8f7462f1 (127.25.137.193:45863)
I20260808 02:48:45.598021 20827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:43277
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:48:45.646178 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41319
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:33965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:45.983223 21140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:45.983511 21140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:45.983649 21140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:45.990995 21140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:45.991124 21140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:45.991264 21140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:48:46.000634 21140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:46.001915 21140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:46.003123 21140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:46.016001 21146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:46.015919 21147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:46.015972 21150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:46.016782 21140 server_base.cc:1034] running on GCE node
I20260808 02:48:46.017448 21140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:46.018392 21140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:46.019599 21140 hybrid_clock.cc:648] HybridClock initialized: now 1786157326019534 us; error 55 us; skew 500 ppm
I20260808 02:48:46.019918 21140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:46 dist-test-slave-k5rr krb5kdc[20759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157326, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:46.022989 21140 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:48:46.024441 21140 webserver.cc:460] Webserver started at http://127.25.137.194:38387/ using document root <none> and password file <none>
I20260808 02:48:46.025112 21140 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:46.025216 21140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:46.025481 21140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:46.028227 21140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b"
format_stamp: "Formatted at 2026-08-08 02:48:46 on dist-test-slave-k5rr"
server_key: "a14208a76d05d2bfe0f8add76bd6e625"
server_key_iv: "d44703ef8e43c323b151b25bf26f613f"
server_key_version: "encryptionkey@0"
I20260808 02:48:46.028934 21140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b"
format_stamp: "Formatted at 2026-08-08 02:48:46 on dist-test-slave-k5rr"
server_key: "a14208a76d05d2bfe0f8add76bd6e625"
server_key_iv: "d44703ef8e43c323b151b25bf26f613f"
server_key_version: "encryptionkey@0"
I20260808 02:48:46.033865 21140 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:48:46.037168 21159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:46.038414 21140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:48:46.038556 21140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b"
format_stamp: "Formatted at 2026-08-08 02:48:46 on dist-test-slave-k5rr"
server_key: "a14208a76d05d2bfe0f8add76bd6e625"
server_key_iv: "d44703ef8e43c323b151b25bf26f613f"
server_key_version: "encryptionkey@0"
I20260808 02:48:46.038740 21140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:46.055210 21140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:46.059237 21140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:46.059507 21140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:46.060446 21140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:46.061815 21140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:46.061892 21140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:46.062031 21140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:46.062083 21140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:46.093647 21140 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:41913
I20260808 02:48:46.093689 21323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:41913 every 8 connection(s)
I20260808 02:48:46.094885 21140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:48:46.096580 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 21140
I20260808 02:48:46.096783 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:48:46.097371 26151 external_mini_cluster.cc:1428] Setting key 8b68228d472ff895cad287fd41fccc0f
Aug 08 02:48:46 dist-test-slave-k5rr krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157326, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:48:46.115650 21328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.097444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:43103 (local address 127.25.137.254:41319)
0808 02:48:46.097893 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:46.097897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:46.099003 (+ 1106us) server_negotiation.cc:407] Connection header received
0808 02:48:46.100899 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:46.100905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:46.101021 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:46.101132 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:46.102578 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.103533 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.105079 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.105638 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.105757 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.109138 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:46.109144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:46.109149 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:46.109186 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:46.112333 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:46.113140 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:46.113146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:46.113149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:46.113299 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:46.113779 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:46.113783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:46.113785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:46.114070 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:46.114197 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:46.115026 (+ 829us) server_negotiation.cc:299] Negotiation successful
0808 02:48:46.115255 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":231,"threads_started":1}
I20260808 02:48:46.116665 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.098037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41319 (local address 127.25.137.194:43103)
0808 02:48:46.098836 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:46.098882 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:46.100629 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:46.101341 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:46.101350 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:46.101846 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:46.102409 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.102427 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.103700 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.103706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.104926 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.104938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.105839 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.105845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.106044 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.106666 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:48:46.106688 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:46.108852 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:46.112503 (+ 3651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:46.112511 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:46.112524 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:46.112982 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:46.113424 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:46.113427 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:46.113430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:46.113651 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:46.114209 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:46.114216 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:46.114643 (+ 427us) client_negotiation.cc:769] Sending connection context
0808 02:48:46.114900 (+ 257us) client_negotiation.cc:240] Negotiation successful
0808 02:48:46.115460 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":322,"threads_started":1}
I20260808 02:48:46.118300 21326 heartbeater.cc:347] Connected to a master server at 127.25.137.254:41319
I20260808 02:48:46.118685 21326 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:46.119549 21326 heartbeater.cc:511] Master 127.25.137.254:41319 requested a full tablet report, sending...
I20260808 02:48:46.121204 20827 ts_manager.cc:194] Registered new tserver with Master: 9bac0dfdeaa54ce0a941e8d5636b4d9b (127.25.137.194:41913)
I20260808 02:48:46.122025 20827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:43103
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:48:46.166199 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41319
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:33965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:46.510187 21334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:46.510485 21334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:46.510646 21334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:46.518253 21334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:46.518376 21334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:46.518556 21334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:48:46.528620 21334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:33965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:41319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:46.529955 21334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:46.531327 21334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:46.545615 21346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:46.545644 21350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:46.545573 21347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:46.546271 21334 server_base.cc:1034] running on GCE node
I20260808 02:48:46.547004 21334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:46.547957 21334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:46.549157 21334 hybrid_clock.cc:648] HybridClock initialized: now 1786157326549106 us; error 47 us; skew 500 ppm
I20260808 02:48:46.549461 21334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:46 dist-test-slave-k5rr krb5kdc[20759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157326, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:46.552642 21334 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:48:46.554193 21334 webserver.cc:460] Webserver started at http://127.25.137.195:34391/ using document root <none> and password file <none>
I20260808 02:48:46.554872 21334 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:46.554991 21334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:46.555294 21334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:46.558171 21334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "45c2bf9bc5714b06b5c09fa254ae67ce"
format_stamp: "Formatted at 2026-08-08 02:48:46 on dist-test-slave-k5rr"
server_key: "b0c2e542881a70132e3524290aa4eb5a"
server_key_iv: "34fcc874cd8e3bfb974923ff1dc7f641"
server_key_version: "encryptionkey@0"
I20260808 02:48:46.558934 21334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "45c2bf9bc5714b06b5c09fa254ae67ce"
format_stamp: "Formatted at 2026-08-08 02:48:46 on dist-test-slave-k5rr"
server_key: "b0c2e542881a70132e3524290aa4eb5a"
server_key_iv: "34fcc874cd8e3bfb974923ff1dc7f641"
server_key_version: "encryptionkey@0"
I20260808 02:48:46.563958 21334 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:48:46.567402 21360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:46.568751 21334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:48:46.568938 21334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "45c2bf9bc5714b06b5c09fa254ae67ce"
format_stamp: "Formatted at 2026-08-08 02:48:46 on dist-test-slave-k5rr"
server_key: "b0c2e542881a70132e3524290aa4eb5a"
server_key_iv: "34fcc874cd8e3bfb974923ff1dc7f641"
server_key_version: "encryptionkey@0"
I20260808 02:48:46.569154 21334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:46.585093 21334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:46.589160 21334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:46.589432 21334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:46.590305 21334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:46.591575 21334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:46.591661 21334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:46.591765 21334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:46.591825 21334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:46.601284 21126 heartbeater.cc:503] Master 127.25.137.254:41319 was elected leader, sending a full tablet report...
I20260808 02:48:46.622845 21334 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:36695
I20260808 02:48:46.622910 21527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:36695 every 8 connection(s)
I20260808 02:48:46.624073 21334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:48:46.625415 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 21334
I20260808 02:48:46.625598 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:48:46.626053 26151 external_mini_cluster.cc:1428] Setting key 9ae8cf68a2305a39041f0e03208ec170
Aug 08 02:48:46 dist-test-slave-k5rr krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157326, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:48:46.643528 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.626560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:34565 (local address 127.25.137.254:41319)
0808 02:48:46.627021 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:46.627026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:46.628169 (+ 1143us) server_negotiation.cc:407] Connection header received
0808 02:48:46.630222 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:46.630226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:46.630326 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:46.630415 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:46.631819 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.632559 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.634101 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.634510 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.634603 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.637878 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:46.637884 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:46.637887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:46.637919 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:46.640621 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:46.641455 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:46.641460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:46.641462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:46.641583 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:46.642007 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:46.642010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:46.642012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:46.642322 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:46.642447 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:46.643085 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 02:48:46.643249 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":228,"threads_started":1}
I20260808 02:48:46.644491 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.627123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:41319 (local address 127.25.137.195:34565)
0808 02:48:46.627987 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:46.628048 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:46.629981 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:46.630601 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:46.630609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:46.631091 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:46.631683 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.631702 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.632707 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.632711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.633990 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.634003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.634680 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.634684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.634882 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.635479 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:48:46.635501 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:46.637657 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:46.640751 (+ 3094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:46.640760 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:46.640773 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:46.641345 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:46.641695 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:46.641699 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:46.641701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:46.641906 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:46.642450 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:46.642456 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:46.642763 (+ 307us) client_negotiation.cc:769] Sending connection context
0808 02:48:46.642994 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 02:48:46.643475 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":331,"threads_started":1}
I20260808 02:48:46.645787 21528 heartbeater.cc:347] Connected to a master server at 127.25.137.254:41319
I20260808 02:48:46.646075 21528 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:46.646775 21528 heartbeater.cc:511] Master 127.25.137.254:41319 requested a full tablet report, sending...
I20260808 02:48:46.648113 20827 ts_manager.cc:194] Registered new tserver with Master: 45c2bf9bc5714b06b5c09fa254ae67ce (127.25.137.195:36695)
I20260808 02:48:46.648965 20827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:34565
I20260808 02:48:46.653797 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:48:46.670614 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.658410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.25.137.254:41319)
0808 02:48:46.658589 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:46.658594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:46.659036 (+ 442us) server_negotiation.cc:407] Connection header received
0808 02:48:46.659226 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:46.659231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:46.659296 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:46.659378 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:46.660871 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.661702 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.663076 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.663461 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.663547 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.665888 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:46.665894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:46.665897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:46.665929 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:46.668313 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:46.668937 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:46.668941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:46.668944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:46.669097 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:46.669473 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:46.669477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:46.669481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:46.669780 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:46.669966 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:46.670215 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 02:48:46.670381 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:48:46.674696 20827 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:48:46.681856 20827 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:48:46.719354 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.694740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:41913 (local address 127.0.0.1:37708)
0808 02:48:46.710245 (+ 15505us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:46.710268 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:46.710429 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:46.711572 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:46.711578 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:46.711614 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:46.712093 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.712108 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.714015 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.714020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.716707 (+ 2687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.716718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.718325 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.718333 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.718505 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.718563 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 02:48:46.718696 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 02:48:46.718874 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15355,"mutex_wait_us":6,"thread_start_us":12091,"threads_started":1}
I20260808 02:48:46.719544 21549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.694778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37878 (local address 127.25.137.195:36695)
0808 02:48:46.710721 (+ 15943us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:46.710730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:46.710765 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:48:46.710868 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:46.710876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:46.711112 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:46.712092 (+ 980us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:46.712379 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.713851 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.716534 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.717807 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.717981 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.718160 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:46.719072 (+ 912us) server_negotiation.cc:299] Negotiation successful
0808 02:48:46.719229 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15765,"thread_start_us":15600,"threads_started":1}
I20260808 02:48:46.719990 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.694496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:36695 (local address 127.0.0.1:37878)
0808 02:48:46.707248 (+ 12752us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:46.707299 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:46.707517 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:46.711372 (+ 3855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:46.711380 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:46.711410 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:46.711856 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.711870 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.714001 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.714006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.716370 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.716386 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.717984 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.717990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.718701 (+ 711us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.718732 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:48:46.718801 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:48:46.718917 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12484,"mutex_wait_us":45,"thread_start_us":16123,"threads_started":1}
I20260808 02:48:46.720101 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.711269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.25.137.193:45863)
0808 02:48:46.711972 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:46.711980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:46.712674 (+ 694us) server_negotiation.cc:407] Connection header received
0808 02:48:46.712823 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:46.712830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:46.713094 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:46.713280 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:46.713818 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.715350 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.717474 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.718804 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.719004 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.719161 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:46.719640 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 02:48:46.719819 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":329,"threads_started":1}
I20260808 02:48:46.720664 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.711973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45863 (local address 127.0.0.1:46156)
0808 02:48:46.712549 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:46.712567 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:46.712689 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:46.713283 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:46.713288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:46.713313 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:46.713650 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.713660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.715492 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.715498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.717308 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.717325 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.719305 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.719312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.719440 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.719471 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:48:46.720345 (+ 874us) client_negotiation.cc:240] Negotiation successful
0808 02:48:46.720442 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":312,"threads_started":1}
I20260808 02:48:46.720682 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.695076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.25.137.194:41913)
0808 02:48:46.706760 (+ 11684us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:46.706771 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:46.710578 (+ 3807us) server_negotiation.cc:407] Connection header received
0808 02:48:46.710710 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:46.710719 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:46.711018 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:46.711253 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:46.712306 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.713754 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.716855 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.718098 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.718254 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.718423 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:46.720202 (+ 1779us) server_negotiation.cc:299] Negotiation successful
0808 02:48:46.720379 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11459,"thread_start_us":11255,"threads_started":1}
I20260808 02:48:46.723742 21431 tablet_service.cc:1467] Processing CreateTablet for tablet 0c56bd081cb0474592ca97db94778607 (DEFAULT_TABLE table=test-table [id=43a094cc55204da8819995c0ffed994a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:46.724131 21231 tablet_service.cc:1467] Processing CreateTablet for tablet 0c56bd081cb0474592ca97db94778607 (DEFAULT_TABLE table=test-table [id=43a094cc55204da8819995c0ffed994a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:46.724131 21021 tablet_service.cc:1467] Processing CreateTablet for tablet 0c56bd081cb0474592ca97db94778607 (DEFAULT_TABLE table=test-table [id=43a094cc55204da8819995c0ffed994a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:46.725201 21431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c56bd081cb0474592ca97db94778607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:46.725487 21021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c56bd081cb0474592ca97db94778607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:46.725514 21231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c56bd081cb0474592ca97db94778607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:46.736114 21554 tablet_bootstrap.cc:492] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce: Bootstrap starting.
I20260808 02:48:46.739789 21560 tablet_bootstrap.cc:492] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: Bootstrap starting.
I20260808 02:48:46.740617 21559 tablet_bootstrap.cc:492] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: Bootstrap starting.
I20260808 02:48:46.741091 21554 tablet_bootstrap.cc:654] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:46.742422 21554 log.cc:824] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:46.743875 21559 tablet_bootstrap.cc:654] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:46.744163 21560 tablet_bootstrap.cc:654] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:46.745097 21559 log.cc:824] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:46.745195 21560 log.cc:824] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:46.746529 21554 tablet_bootstrap.cc:492] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce: No bootstrap required, opened a new log
I20260808 02:48:46.746826 21554 ts_tablet_manager.cc:1397] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260808 02:48:46.751713 21559 tablet_bootstrap.cc:492] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: No bootstrap required, opened a new log
I20260808 02:48:46.751965 21560 tablet_bootstrap.cc:492] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: No bootstrap required, opened a new log
I20260808 02:48:46.751987 21559 ts_tablet_manager.cc:1397] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260808 02:48:46.752204 21560 ts_tablet_manager.cc:1397] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260808 02:48:46.757135 21554 raft_consensus.cc:348] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36695 } } peers { permanent_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 41913 } } peers { permanent_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45863 } }
I20260808 02:48:46.757644 21554 raft_consensus.cc:374] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:46.757752 21554 raft_consensus.cc:729] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c2bf9bc5714b06b5c09fa254ae67ce, State: Initialized, Role: FOLLOWER
I20260808 02:48:46.758359 21554 consensus_queue.cc:260] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36695 } } peers { permanent_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 41913 } } peers { permanent_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45863 } }
I20260808 02:48:46.759460 21528 heartbeater.cc:503] Master 127.25.137.254:41319 was elected leader, sending a full tablet report...
I20260808 02:48:46.760056 21554 ts_tablet_manager.cc:1428] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260808 02:48:46.760529 21560 raft_consensus.cc:348] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36695 } } peers { permanent_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 41913 } } peers { permanent_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45863 } }
I20260808 02:48:46.760934 21560 raft_consensus.cc:374] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:46.761081 21560 raft_consensus.cc:729] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bac0dfdeaa54ce0a941e8d5636b4d9b, State: Initialized, Role: FOLLOWER
I20260808 02:48:46.761619 21560 consensus_queue.cc:260] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36695 } } peers { permanent_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 41913 } } peers { permanent_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45863 } }
I20260808 02:48:46.762665 21326 heartbeater.cc:503] Master 127.25.137.254:41319 was elected leader, sending a full tablet report...
I20260808 02:48:46.762558 21559 raft_consensus.cc:348] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36695 } } peers { permanent_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 41913 } } peers { permanent_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45863 } }
I20260808 02:48:46.762960 21559 raft_consensus.cc:374] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:46.763056 21559 raft_consensus.cc:729] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd0cc1df317c4f118b250e8c8f7462f1, State: Initialized, Role: FOLLOWER
I20260808 02:48:46.763262 21560 ts_tablet_manager.cc:1428] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260808 02:48:46.763667 21559 consensus_queue.cc:260] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36695 } } peers { permanent_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 41913 } } peers { permanent_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45863 } }
I20260808 02:48:46.765314 21559 ts_tablet_manager.cc:1428] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260808 02:48:46.788906 21565 raft_consensus.cc:482] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:48:46.789186 21565 raft_consensus.cc:504] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36695 } } peers { permanent_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 41913 } } peers { permanent_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45863 } }
I20260808 02:48:46.790817 21565 leader_election.cc:288] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bac0dfdeaa54ce0a941e8d5636b4d9b (127.25.137.194:41913), bd0cc1df317c4f118b250e8c8f7462f1 (127.25.137.193:45863)
I20260808 02:48:46.796862 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.791269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:54201 (local address 127.25.137.194:41913)
0808 02:48:46.791455 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:46.791461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:46.791483 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:48:46.791657 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:46.791661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:46.791723 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:46.791835 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:46.792229 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.793251 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.795022 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.796034 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.796142 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.796192 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:46.796567 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 02:48:46.796651 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:48:46.796962 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.791005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:41913 (local address 127.25.137.195:54201)
0808 02:48:46.791329 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:46.791350 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:46.791474 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:46.791852 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:46.791857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:46.791882 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:46.792074 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.792083 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.793398 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.793404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.794868 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.794881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.796185 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.796191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.796409 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.796445 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:48:46.796546 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 02:48:46.796676 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260808 02:48:46.797680 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.791340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:57805 (local address 127.25.137.193:45863)
0808 02:48:46.791549 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:46.791554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:46.792148 (+ 594us) server_negotiation.cc:407] Connection header received
0808 02:48:46.792268 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:46.792272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:46.792333 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:46.792396 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:46.792953 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.793858 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.795699 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.796731 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.796881 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.796945 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:46.797326 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 02:48:46.797446 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:48:46.797703 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.791268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45863 (local address 127.25.137.195:57805)
0808 02:48:46.792043 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:46.792057 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:46.792161 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:46.792431 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:46.792436 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:46.792460 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:46.792790 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.792803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.793999 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.794003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.795540 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:46.795555 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.796964 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.796970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:46.797152 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.797184 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:48:46.797274 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 02:48:46.797390 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":428,"threads_started":1}
I20260808 02:48:46.798002 21260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c56bd081cb0474592ca97db94778607" candidate_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" is_pre_election: true
I20260808 02:48:46.798540 21260 raft_consensus.cc:2454] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c2bf9bc5714b06b5c09fa254ae67ce in term 0.
I20260808 02:48:46.798652 21053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c56bd081cb0474592ca97db94778607" candidate_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" is_pre_election: true
I20260808 02:48:46.799202 21053 raft_consensus.cc:2454] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c2bf9bc5714b06b5c09fa254ae67ce in term 0.
I20260808 02:48:46.799283 21361 leader_election.cc:302] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c2bf9bc5714b06b5c09fa254ae67ce, 9bac0dfdeaa54ce0a941e8d5636b4d9b; no voters:
I20260808 02:48:46.799681 21565 raft_consensus.cc:2781] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:48:46.799868 21565 raft_consensus.cc:482] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:48:46.799984 21565 raft_consensus.cc:3037] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:46.802225 21565 raft_consensus.cc:504] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36695 } } peers { permanent_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 41913 } } peers { permanent_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45863 } }
I20260808 02:48:46.802778 21565 leader_election.cc:288] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [CANDIDATE]: Term 1 election: Requested vote from peers 9bac0dfdeaa54ce0a941e8d5636b4d9b (127.25.137.194:41913), bd0cc1df317c4f118b250e8c8f7462f1 (127.25.137.193:45863)
I20260808 02:48:46.803265 21053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c56bd081cb0474592ca97db94778607" candidate_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0cc1df317c4f118b250e8c8f7462f1"
I20260808 02:48:46.803265 21260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c56bd081cb0474592ca97db94778607" candidate_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b"
I20260808 02:48:46.803471 21053 raft_consensus.cc:3037] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:46.803472 21260 raft_consensus.cc:3037] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:46.806582 21260 raft_consensus.cc:2454] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c2bf9bc5714b06b5c09fa254ae67ce in term 1.
I20260808 02:48:46.806582 21053 raft_consensus.cc:2454] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c2bf9bc5714b06b5c09fa254ae67ce in term 1.
I20260808 02:48:46.807021 21363 leader_election.cc:302] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 45c2bf9bc5714b06b5c09fa254ae67ce, 9bac0dfdeaa54ce0a941e8d5636b4d9b, bd0cc1df317c4f118b250e8c8f7462f1; no voters:
I20260808 02:48:46.807377 21565 raft_consensus.cc:2781] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:48:46.807821 21565 raft_consensus.cc:686] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 1 LEADER]: Becoming Leader. State: Replica: 45c2bf9bc5714b06b5c09fa254ae67ce, State: Running, Role: LEADER
I20260808 02:48:46.808254 21565 consensus_queue.cc:237] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36695 } } peers { permanent_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 41913 } } peers { permanent_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45863 } }
I20260808 02:48:46.812520 20827 catalog_manager.cc:5495] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 45c2bf9bc5714b06b5c09fa254ae67ce (127.25.137.195). New cstate: current_term: 1 leader_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c2bf9bc5714b06b5c09fa254ae67ce" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 41913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45863 } health_report { overall_health: UNKNOWN } } }
W20260808 02:48:46.822117 21127 tablet.cc:2369] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:48:46.840617 21549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.835306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37890 (local address 127.25.137.195:36695)
0808 02:48:46.835515 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:46.835520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:46.835634 (+ 114us) server_negotiation.cc:407] Connection header received
0808 02:48:46.835846 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:46.835858 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:46.835938 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:46.836066 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:48:46.836452 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.837211 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.838607 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.838999 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.839106 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.839584 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:48:46.840055 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:48:46.840184 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:46.840283 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 02:48:46.840372 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 02:48:46.846726 21327 tablet.cc:2369] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:48:46.849201 21260 raft_consensus.cc:1261] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b [term 1 FOLLOWER]: Refusing update from remote peer 45c2bf9bc5714b06b5c09fa254ae67ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:48:46.849201 21053 raft_consensus.cc:1261] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1 [term 1 FOLLOWER]: Refusing update from remote peer 45c2bf9bc5714b06b5c09fa254ae67ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:48:46.850246 21565 consensus_queue.cc:1035] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bac0dfdeaa54ce0a941e8d5636b4d9b" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 41913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:48:46.850608 21570 consensus_queue.cc:1035] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd0cc1df317c4f118b250e8c8f7462f1" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:48:46.862870 21578 mvcc.cc:205] Tried to move back new op lower bound from 7316100410762911744 to 7316100410610495488. Current Snapshot: MvccSnapshot[applied={T|T < 7316100410762911744}]
I20260808 02:48:46.878345 20827 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:48:46.878659 20827 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:48:46.882705 20827 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d27e43abc6540f2b58540d3c2df695e: Sending DeleteTablet for 3 replicas of tablet 0c56bd081cb0474592ca97db94778607
I20260808 02:48:46.884052 21021 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c56bd081cb0474592ca97db94778607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:46 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:46156
I20260808 02:48:46.884052 21231 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c56bd081cb0474592ca97db94778607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:46 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:37708
I20260808 02:48:46.884243 21431 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c56bd081cb0474592ca97db94778607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:46 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:37878
I20260808 02:48:46.886277 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 20924
I20260808 02:48:46.897586 21593 tablet_replica.cc:330] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce: stopping tablet replica
I20260808 02:48:46.898075 21593 raft_consensus.cc:2229] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:48:46.898888 21593 raft_consensus.cc:2258] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:46.901181 21593 ts_tablet_manager.cc:1905] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:46.904204 21590 tablet_replica.cc:330] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: stopping tablet replica
I20260808 02:48:46.904649 21590 raft_consensus.cc:2229] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:48:46.905277 21590 raft_consensus.cc:2258] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:46.909492 21592 tablet_replica.cc:330] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: stopping tablet replica
I20260808 02:48:46.909977 21592 raft_consensus.cc:2229] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:48:46.910524 21592 raft_consensus.cc:2258] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:46.912145 21590 ts_tablet_manager.cc:1905] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:46.912657 21592 ts_tablet_manager.cc:1905] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 02:48:46 dist-test-slave-k5rr krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:48:46.920011 21592 ts_tablet_manager.cc:1918] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:46.920280 21592 log.cc:1196] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/0c56bd081cb0474592ca97db94778607
I20260808 02:48:46.920518 21590 ts_tablet_manager.cc:1918] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:46.920706 21590 log.cc:1196] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/0c56bd081cb0474592ca97db94778607
I20260808 02:48:46.921161 21592 ts_tablet_manager.cc:1939] T 0c56bd081cb0474592ca97db94778607 P bd0cc1df317c4f118b250e8c8f7462f1: Deleting consensus metadata
I20260808 02:48:46.921324 21590 ts_tablet_manager.cc:1939] T 0c56bd081cb0474592ca97db94778607 P 9bac0dfdeaa54ce0a941e8d5636b4d9b: Deleting consensus metadata
I20260808 02:48:46.922466 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:46.887750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.25.137.193:45863)
0808 02:48:46.887955 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:46.887961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:46.897837 (+ 9876us) server_negotiation.cc:407] Connection header received
0808 02:48:46.898161 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:46.898168 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:46.898260 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:46.898343 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:46.900417 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.901656 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.907898 (+ 6242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:46.908630 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:46.908761 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:46.914809 (+ 6048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:46.914824 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:46.914845 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:46.914886 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:46.918773 (+ 3887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:46.919720 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:46.919729 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:46.919732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:46.919907 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:46.920409 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:46.920414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:46.920417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:46.920901 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:46.921225 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:46.921843 (+ 618us) server_negotiation.cc:299] Negotiation successful
0808 02:48:46.922061 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:48:46.923306 20804 catalog_manager.cc:4841] TS 9bac0dfdeaa54ce0a941e8d5636b4d9b (127.25.137.194:41913): tablet 0c56bd081cb0474592ca97db94778607 (table test-table [id=43a094cc55204da8819995c0ffed994a]) successfully deleted
I20260808 02:48:46.923369 20805 catalog_manager.cc:4841] TS bd0cc1df317c4f118b250e8c8f7462f1 (127.25.137.193:45863): tablet 0c56bd081cb0474592ca97db94778607 (table test-table [id=43a094cc55204da8819995c0ffed994a]) successfully deleted
I20260808 02:48:46.923950 21118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:46.924680 21593 ts_tablet_manager.cc:1918] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:46.924911 21593 log.cc:1196] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/0c56bd081cb0474592ca97db94778607
I20260808 02:48:46.925571 21593 ts_tablet_manager.cc:1939] T 0c56bd081cb0474592ca97db94778607 P 45c2bf9bc5714b06b5c09fa254ae67ce: Deleting consensus metadata
I20260808 02:48:46.927445 20805 catalog_manager.cc:4841] TS 45c2bf9bc5714b06b5c09fa254ae67ce (127.25.137.195:36695): tablet 0c56bd081cb0474592ca97db94778607 (table test-table [id=43a094cc55204da8819995c0ffed994a]) successfully deleted
I20260808 02:48:47.031826 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 20924
I20260808 02:48:47.056911 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 21140
Aug 08 02:48:47 dist-test-slave-k5rr krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:48:47.073454 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:47.057789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.25.137.194:41913)
0808 02:48:47.058006 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:47.058012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:47.058033 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:48:47.058165 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:47.058171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:47.058242 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:47.058325 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:47.059618 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:47.060348 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:47.061679 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:47.062091 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:47.062207 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:47.066792 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:47.066814 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:47.066835 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:47.066884 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:47.070415 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:47.071108 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:47.071117 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:47.071121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:47.071274 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:47.071622 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:47.071627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:47.071631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:47.072144 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:47.072452 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:47.072816 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 02:48:47.073073 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 02:48:47.074227 21318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:47.162622 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 21140
I20260808 02:48:47.194367 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 21334
Aug 08 02:48:47 dist-test-slave-k5rr krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:48:47.210285 21549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:47.195261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37904 (local address 127.25.137.195:36695)
0808 02:48:47.195458 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:47.195464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:47.195482 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:48:47.195543 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:47.195547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:47.195606 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:47.195694 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:47.197046 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:47.197904 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:47.199168 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:47.199631 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:47.199745 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:47.204354 (+ 4609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:47.204367 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:47.204383 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:47.204420 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:47.207394 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:47.208086 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:47.208094 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:47.208096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:47.208246 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:47.208638 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:47.208642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:47.208644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:47.209081 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:47.209381 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:47.209759 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 02:48:47.209965 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:48:47.211066 21520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:47.334870 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 21334
I20260808 02:48:47.361434 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 20780
I20260808 02:48:47.362586 20876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:47.473862 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 20780
2026-08-08T02:48:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4532 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T02:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:48:47Z Disabled control of system clock
I20260808 02:48:47.571273 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45533
--webserver_interface=127.25.137.254
--webserver_port=0
--builtin_ntp_servers=127.25.137.212:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:45533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:48:47.908618 21607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:47.908908 21607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:47.909042 21607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:47.916226 21607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:48:47.916327 21607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:47.916369 21607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:47.916447 21607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:48:47.916500 21607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:48:47.925768 21607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:45533
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:47.927150 21607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:47.929347 21607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:47.941129 21614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:47.941082 21616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:47.941171 21618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:47.941982 21607 server_base.cc:1034] running on GCE node
I20260808 02:48:47.942759 21607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:47.944144 21607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:47.945353 21607 hybrid_clock.cc:648] HybridClock initialized: now 1786157327945303 us; error 38 us; skew 500 ppm
I20260808 02:48:47.945708 21607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:48:47.948033 21607 webserver.cc:460] Webserver started at http://127.25.137.254:33901/ using document root <none> and password file <none>
I20260808 02:48:47.948735 21607 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:47.948828 21607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:47.949182 21607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:47.952082 21607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "4841ea0bede44e51aa96b56a0fb5d57a"
format_stamp: "Formatted at 2026-08-08 02:48:47 on dist-test-slave-k5rr"
server_key: "a2050687ecc4d8cba8a57ec7f374c248"
server_key_iv: "0bf983e61b6f09b1d9100cc856a6dd35"
server_key_version: "encryptionkey@0"
I20260808 02:48:47.952912 21607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "4841ea0bede44e51aa96b56a0fb5d57a"
format_stamp: "Formatted at 2026-08-08 02:48:47 on dist-test-slave-k5rr"
server_key: "a2050687ecc4d8cba8a57ec7f374c248"
server_key_iv: "0bf983e61b6f09b1d9100cc856a6dd35"
server_key_version: "encryptionkey@0"
I20260808 02:48:47.957981 21607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 02:48:47.961321 21631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:47.962659 21607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:48:47.962828 21607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "4841ea0bede44e51aa96b56a0fb5d57a"
format_stamp: "Formatted at 2026-08-08 02:48:47 on dist-test-slave-k5rr"
server_key: "a2050687ecc4d8cba8a57ec7f374c248"
server_key_iv: "0bf983e61b6f09b1d9100cc856a6dd35"
server_key_version: "encryptionkey@0"
I20260808 02:48:47.963020 21607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:47.986384 21607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:47.987241 21607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:47.987509 21607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:48.005458 21607 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:45533
I20260808 02:48:48.005493 21715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:45533 every 8 connection(s)
I20260808 02:48:48.006750 21607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:48:48.008718 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 21607
I20260808 02:48:48.008880 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:48:48.009363 26151 external_mini_cluster.cc:1428] Setting key 882f2cadc6eef2e1828f54edd95ee862
I20260808 02:48:48.010694 21716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:48.022465 21722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:48.011223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46258 (local address 127.25.137.254:45533)
0808 02:48:48.012207 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:48.012218 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:48.012261 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:48:48.013896 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:48.013917 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:48.014371 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:48.014735 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:48.015280 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:48.016636 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:48.018150 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:48.018814 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:48.018982 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:48.020184 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:48.020191 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:48.020207 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:48.020774 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:48.020855 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:48.020877 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:48.020982 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:48.021225 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 02:48:48.021521 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":347,"threads_started":1}
I20260808 02:48:48.022753 21716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a: Bootstrap starting.
I20260808 02:48:48.026001 21716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:48.026973 21716 log.cc:824] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:48.029829 21716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a: No bootstrap required, opened a new log
I20260808 02:48:48.037496 21716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4841ea0bede44e51aa96b56a0fb5d57a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45533 } }
I20260808 02:48:48.037860 21716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:48.037959 21716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4841ea0bede44e51aa96b56a0fb5d57a, State: Initialized, Role: FOLLOWER
I20260808 02:48:48.038498 21716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4841ea0bede44e51aa96b56a0fb5d57a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45533 } }
I20260808 02:48:48.038710 21716 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:48:48.038841 21716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:48:48.038998 21716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:48.041530 21716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4841ea0bede44e51aa96b56a0fb5d57a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45533 } }
I20260808 02:48:48.042121 21716 leader_election.cc:302] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4841ea0bede44e51aa96b56a0fb5d57a; no voters:
I20260808 02:48:48.042693 21716 leader_election.cc:288] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:48:48.042882 21725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:48:48.043547 21725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [term 1 LEADER]: Becoming Leader. State: Replica: 4841ea0bede44e51aa96b56a0fb5d57a, State: Running, Role: LEADER
I20260808 02:48:48.044068 21725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4841ea0bede44e51aa96b56a0fb5d57a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45533 } }
I20260808 02:48:48.044744 21716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:48:48.046727 21726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4841ea0bede44e51aa96b56a0fb5d57a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4841ea0bede44e51aa96b56a0fb5d57a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45533 } } }
I20260808 02:48:48.046752 21728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4841ea0bede44e51aa96b56a0fb5d57a. Latest consensus state: current_term: 1 leader_uuid: "4841ea0bede44e51aa96b56a0fb5d57a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4841ea0bede44e51aa96b56a0fb5d57a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45533 } } }
I20260808 02:48:48.047187 21726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:48.047191 21728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:48.048794 21732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:48:48.054219 21732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:48:48.064165 21732 catalog_manager.cc:1347] Generated new cluster ID: 402bb2caa6904b5a866669755eb294e2
I20260808 02:48:48.064306 21732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:48:48.080940 21732 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:48:48.082795 21732 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:48:48.094853 21732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a: Generated new TSK 0
I20260808 02:48:48.095935 21732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:48:48.105175 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45533
--builtin_ntp_servers=127.25.137.212:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:48:48.469280 21754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:48.469560 21754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:48.469693 21754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:48.476931 21754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:48.477049 21754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:48.477192 21754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:48:48.486394 21754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:48.487670 21754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:48.489542 21754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:48.502146 21764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:48.502107 21768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:48.502189 21765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:48.503166 21754 server_base.cc:1034] running on GCE node
I20260808 02:48:48.503787 21754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:48.504812 21754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:48.506045 21754 hybrid_clock.cc:648] HybridClock initialized: now 1786157328505983 us; error 42 us; skew 500 ppm
I20260808 02:48:48.506392 21754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:48:48.508766 21754 webserver.cc:460] Webserver started at http://127.25.137.193:43861/ using document root <none> and password file <none>
I20260808 02:48:48.509522 21754 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:48.509621 21754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:48.509907 21754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:48.513413 21754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "4f16d50d5a234bf08856d88e6bdf30cb"
format_stamp: "Formatted at 2026-08-08 02:48:48 on dist-test-slave-k5rr"
server_key: "f175bc5b7249eb43dc44fa304b22181a"
server_key_iv: "d8da18944ff6dd8ee4f95e8a60533557"
server_key_version: "encryptionkey@0"
I20260808 02:48:48.514205 21754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "4f16d50d5a234bf08856d88e6bdf30cb"
format_stamp: "Formatted at 2026-08-08 02:48:48 on dist-test-slave-k5rr"
server_key: "f175bc5b7249eb43dc44fa304b22181a"
server_key_iv: "d8da18944ff6dd8ee4f95e8a60533557"
server_key_version: "encryptionkey@0"
I20260808 02:48:48.519692 21754 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:48:48.523422 21775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:48.524827 21754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:48:48.524997 21754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "4f16d50d5a234bf08856d88e6bdf30cb"
format_stamp: "Formatted at 2026-08-08 02:48:48 on dist-test-slave-k5rr"
server_key: "f175bc5b7249eb43dc44fa304b22181a"
server_key_iv: "d8da18944ff6dd8ee4f95e8a60533557"
server_key_version: "encryptionkey@0"
I20260808 02:48:48.525275 21754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:48.545490 21754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:48.546353 21754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:48.546675 21754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:48.547680 21754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:48.549526 21754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:48.549640 21754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:48.549760 21754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:48.549866 21754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:48.582228 21754 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:45811
I20260808 02:48:48.582285 21940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:45811 every 8 connection(s)
I20260808 02:48:48.583645 21754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:48:48.590519 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 21754
I20260808 02:48:48.590703 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:48:48.591220 26151 external_mini_cluster.cc:1428] Setting key db5f96715863c169f66ed01a61083230
I20260808 02:48:48.594199 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45533
--builtin_ntp_servers=127.25.137.212:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:48:48.595856 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:48.586249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:53215 (local address 127.25.137.254:45533)
0808 02:48:48.586729 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:48.586734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:48.587690 (+ 956us) server_negotiation.cc:407] Connection header received
0808 02:48:48.589548 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:48.589554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:48.589643 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:48.589740 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:48.590847 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:48.591624 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:48.593168 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:48.593570 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:48.593682 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:48.595021 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:48.595026 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:48.595029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:48.595052 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:48.595104 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:48.595117 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:48.595193 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:48.595407 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 02:48:48.595504 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":253,"threads_started":1}
I20260808 02:48:48.596778 21947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:48.586650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45533 (local address 127.25.137.193:53215)
0808 02:48:48.587531 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:48.587567 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:48.589269 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:48.589985 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:48.589994 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:48.590081 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:48.590666 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:48.590689 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:48.591779 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:48.591783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:48.593027 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:48.593044 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:48.593803 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:48.593811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:48.594038 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:48.594705 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:48:48.594708 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:48.594746 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:48:48.594835 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:48.595209 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:48.595221 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:48.595228 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:48:48.595389 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 02:48:48.595762 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":267,"threads_started":1}
I20260808 02:48:48.598474 21941 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45533
I20260808 02:48:48.598809 21941 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:48.599736 21941 heartbeater.cc:511] Master 127.25.137.254:45533 requested a full tablet report, sending...
I20260808 02:48:48.602180 21657 ts_manager.cc:194] Registered new tserver with Master: 4f16d50d5a234bf08856d88e6bdf30cb (127.25.137.193:45811)
I20260808 02:48:48.604108 21657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.193:53215
W20260808 02:48:48.927256 21948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:48.927539 21948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:48.927665 21948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:48.934872 21948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:48.934968 21948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:48.935112 21948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:48:48.944248 21948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:48.945649 21948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:48.947505 21948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:48.960246 21954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:48.960311 21956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:48.960345 21958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:48.961162 21948 server_base.cc:1034] running on GCE node
I20260808 02:48:48.961733 21948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:48.962641 21948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:48.963883 21948 hybrid_clock.cc:648] HybridClock initialized: now 1786157328963816 us; error 58 us; skew 500 ppm
I20260808 02:48:48.964214 21948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:48:48.966622 21948 webserver.cc:460] Webserver started at http://127.25.137.194:45369/ using document root <none> and password file <none>
I20260808 02:48:48.967322 21948 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:48.967433 21948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:48.967742 21948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:48.970602 21948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "3500bff31aac4efeb4a1fd7a959ac7e6"
format_stamp: "Formatted at 2026-08-08 02:48:48 on dist-test-slave-k5rr"
server_key: "2e6771a9a1afd2a21c9d7adcd2974f58"
server_key_iv: "cc005304e781e4025faaa70e9626c003"
server_key_version: "encryptionkey@0"
I20260808 02:48:48.971385 21948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "3500bff31aac4efeb4a1fd7a959ac7e6"
format_stamp: "Formatted at 2026-08-08 02:48:48 on dist-test-slave-k5rr"
server_key: "2e6771a9a1afd2a21c9d7adcd2974f58"
server_key_iv: "cc005304e781e4025faaa70e9626c003"
server_key_version: "encryptionkey@0"
I20260808 02:48:48.976305 21948 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:48:48.980191 21969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:48.981515 21948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:48:48.981673 21948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "3500bff31aac4efeb4a1fd7a959ac7e6"
format_stamp: "Formatted at 2026-08-08 02:48:48 on dist-test-slave-k5rr"
server_key: "2e6771a9a1afd2a21c9d7adcd2974f58"
server_key_iv: "cc005304e781e4025faaa70e9626c003"
server_key_version: "encryptionkey@0"
I20260808 02:48:48.981877 21948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:49.006085 21948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:49.006917 21948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:49.007216 21948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:49.008206 21948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:49.009724 21948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:49.009831 21948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:49.009935 21948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:49.010007 21948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:49.040416 21948 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:34563
I20260808 02:48:49.040484 22133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:34563 every 8 connection(s)
I20260808 02:48:49.041715 21948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:48:49.045657 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 21948
I20260808 02:48:49.045849 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:48:49.046455 26151 external_mini_cluster.cc:1428] Setting key 044d5b838b85f88836b750f6f8bd6572
I20260808 02:48:49.049577 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45533
--builtin_ntp_servers=127.25.137.212:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:48:49.054979 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.044123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:51371 (local address 127.25.137.254:45533)
0808 02:48:49.044289 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:49.044295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:49.045713 (+ 1418us) server_negotiation.cc:407] Connection header received
0808 02:48:49.047892 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:49.047897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:49.047980 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:49.048062 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:49.049382 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.050163 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.052328 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.052716 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.052821 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.054289 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:49.054294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:49.054298 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:49.054317 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:49.054349 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:49.054361 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:49.054428 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:49.054663 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 02:48:49.054743 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:48:49.056216 22137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.044527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45533 (local address 127.25.137.194:51371)
0808 02:48:49.045534 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:49.045586 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:49.047614 (+ 2028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:49.048293 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:49.048302 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:49.048400 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:49.049202 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.049229 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.050335 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.050340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.052179 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.052194 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.052941 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.052947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.053240 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.053932 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:48:49.053935 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:49.053982 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:48:49.054079 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:49.054449 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:49.054458 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:49.054465 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:48:49.054628 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 02:48:49.055004 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":283,"threads_started":1}
I20260808 02:48:49.057577 22134 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45533
I20260808 02:48:49.057858 22134 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:49.058576 22134 heartbeater.cc:511] Master 127.25.137.254:45533 requested a full tablet report, sending...
I20260808 02:48:49.059906 21657 ts_manager.cc:194] Registered new tserver with Master: 3500bff31aac4efeb4a1fd7a959ac7e6 (127.25.137.194:34563)
I20260808 02:48:49.060694 21657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.194:51371
W20260808 02:48:49.384231 22138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:49.384505 22138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:49.384629 22138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:49.392010 22138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:49.392122 22138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:49.392258 22138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:48:49.401382 22138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:49.402658 22138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:49.404397 22138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:49.417131 22146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:49.417155 22145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:49.416967 22150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:49.418166 22138 server_base.cc:1034] running on GCE node
I20260808 02:48:49.418695 22138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:49.419596 22138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:49.420842 22138 hybrid_clock.cc:648] HybridClock initialized: now 1786157329420790 us; error 42 us; skew 500 ppm
I20260808 02:48:49.421178 22138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:48:49.423344 22138 webserver.cc:460] Webserver started at http://127.25.137.195:44943/ using document root <none> and password file <none>
I20260808 02:48:49.424006 22138 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:49.424093 22138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:49.424361 22138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:49.427583 22138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "fdcaef5ad46641ce97d298c881cddb30"
format_stamp: "Formatted at 2026-08-08 02:48:49 on dist-test-slave-k5rr"
server_key: "343d59dbb93e5d57ea037ec404b4601e"
server_key_iv: "20e5df85473c612b530543136bde897b"
server_key_version: "encryptionkey@0"
I20260808 02:48:49.428427 22138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "fdcaef5ad46641ce97d298c881cddb30"
format_stamp: "Formatted at 2026-08-08 02:48:49 on dist-test-slave-k5rr"
server_key: "343d59dbb93e5d57ea037ec404b4601e"
server_key_iv: "20e5df85473c612b530543136bde897b"
server_key_version: "encryptionkey@0"
I20260808 02:48:49.433895 22138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:48:49.437373 22162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:49.438655 22138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:48:49.438802 22138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "fdcaef5ad46641ce97d298c881cddb30"
format_stamp: "Formatted at 2026-08-08 02:48:49 on dist-test-slave-k5rr"
server_key: "343d59dbb93e5d57ea037ec404b4601e"
server_key_iv: "20e5df85473c612b530543136bde897b"
server_key_version: "encryptionkey@0"
I20260808 02:48:49.439000 22138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:49.458799 22138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:49.459662 22138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:49.459970 22138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:49.460976 22138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:49.462604 22138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:49.462702 22138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:49.462792 22138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:49.462854 22138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:49.498328 22138 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:45521
I20260808 02:48:49.498384 22351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:45521 every 8 connection(s)
I20260808 02:48:49.499725 22138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:48:49.501375 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 22138
I20260808 02:48:49.501536 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:48:49.501996 26151 external_mini_cluster.cc:1428] Setting key 1e1773f19314777dc02954ee2e9e4a34
I20260808 02:48:49.512235 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.502349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:52809 (local address 127.25.137.254:45533)
0808 02:48:49.502577 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:49.502584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:49.504096 (+ 1512us) server_negotiation.cc:407] Connection header received
0808 02:48:49.506312 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:49.506316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:49.506382 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:49.506461 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:49.507448 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.508178 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.509822 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.510228 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.510322 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.511637 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:49.511641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:49.511644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:49.511660 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:49.511687 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:49.511698 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:49.511762 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:49.511923 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 02:48:49.512000 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:48:49.513199 22355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.502846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45533 (local address 127.25.137.195:52809)
0808 02:48:49.503918 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:49.503973 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:49.506063 (+ 2090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:49.506634 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:49.506642 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:49.506725 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:49.507299 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.507318 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.508336 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.508340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.509679 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.509692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.510458 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.510464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.510666 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.511354 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:48:49.511357 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:49.511393 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:48:49.511475 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:49.511767 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:49.511773 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:49.511778 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:48:49.511910 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 02:48:49.512188 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":344,"threads_started":1}
I20260808 02:48:49.514535 22353 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45533
I20260808 02:48:49.514868 22353 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:49.515580 22353 heartbeater.cc:511] Master 127.25.137.254:45533 requested a full tablet report, sending...
I20260808 02:48:49.517232 21657 ts_manager.cc:194] Registered new tserver with Master: fdcaef5ad46641ce97d298c881cddb30 (127.25.137.195:45521)
I20260808 02:48:49.518070 21657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.195:52809
I20260808 02:48:49.529983 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:48:49.540059 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.534284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46266 (local address 127.25.137.254:45533)
0808 02:48:49.534499 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:49.534506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:49.534930 (+ 424us) server_negotiation.cc:407] Connection header received
0808 02:48:49.535092 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:49.535098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:49.535177 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:49.535257 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:49.535749 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.536454 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.537832 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.538250 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.538369 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.539416 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:49.539421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:49.539424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:49.539441 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:49.539472 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:49.539484 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:49.539554 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:49.539684 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 02:48:49.539771 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:48:49.543968 21657 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:48:49.551371 21657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:48:49.585688 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.563749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:34563 (local address 127.0.0.1:46042)
0808 02:48:49.564699 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:49.564745 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:49.564909 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:49.577781 (+ 12872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:49.577789 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:49.577830 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:49.578193 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.578208 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.579793 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.579798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.581454 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.581471 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.584680 (+ 3209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.584686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.584894 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.584956 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 02:48:49.585110 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 02:48:49.585323 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":33,"thread_start_us":13103,"threads_started":1}
I20260808 02:48:49.586388 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.566137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:45521 (local address 127.0.0.1:40626)
0808 02:48:49.577124 (+ 10987us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:49.577144 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:49.577286 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:49.578527 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:49.578533 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:49.578555 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:49.579035 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.579048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.581264 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.581269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.583415 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.583429 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.585245 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.585251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.585407 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.585440 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:48:49.585503 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 02:48:49.585689 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10852,"mutex_wait_us":5,"thread_start_us":9277,"threads_started":1}
I20260808 02:48:49.586503 22372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.563988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46042 (local address 127.25.137.194:34563)
0808 02:48:49.576746 (+ 12758us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:49.576756 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:49.576794 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:48:49.576921 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:49.576931 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:49.577253 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:49.577471 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:49.578425 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.579663 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.581706 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.584188 (+ 2482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.584343 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.584553 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:49.585971 (+ 1418us) server_negotiation.cc:299] Negotiation successful
0808 02:48:49.586165 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12569,"thread_start_us":12438,"threads_started":1}
I20260808 02:48:49.587148 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.564033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53742)
0808 02:48:49.575805 (+ 11772us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:49.575830 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:49.575984 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:49.578253 (+ 2269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:49.578263 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:49.578290 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:49.578833 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.578848 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.581318 (+ 2470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.581325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.583661 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.583679 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.586204 (+ 2525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.586212 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.586447 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.586485 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 02:48:49.586573 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 02:48:49.586700 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11587,"mutex_wait_us":25,"thread_start_us":640,"threads_started":1}
I20260808 02:48:49.587999 22378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.566744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.25.137.195:45521)
0808 02:48:49.577908 (+ 11164us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:49.577917 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:49.577951 (+ 34us) server_negotiation.cc:407] Connection header received
0808 02:48:49.578047 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:49.578055 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:49.578281 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:49.579261 (+ 980us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:49.579552 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.581111 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.583649 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.585042 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.587237 (+ 2195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.587436 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:49.587490 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 02:48:49.587636 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10982,"thread_start_us":10809,"threads_started":1}
I20260808 02:48:49.588239 22373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.564546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53742 (local address 127.25.137.193:45811)
0808 02:48:49.577558 (+ 13012us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:49.577568 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:49.577600 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:48:49.577706 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:49.577714 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:49.577971 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:49.579258 (+ 1287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:49.579500 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.581111 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.583920 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.585457 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.585687 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.587565 (+ 1878us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:49.587651 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 02:48:49.587836 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12825,"thread_start_us":14439,"threads_started":1}
I20260808 02:48:49.590703 22043 tablet_service.cc:1467] Processing CreateTablet for tablet a4379a5410d346489a49ce44fd10a21a (DEFAULT_TABLE table=test-table [id=e7816306f3c849d68311e62294213d38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:49.592209 22043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4379a5410d346489a49ce44fd10a21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:49.591761 21848 tablet_service.cc:1467] Processing CreateTablet for tablet a4379a5410d346489a49ce44fd10a21a (DEFAULT_TABLE table=test-table [id=e7816306f3c849d68311e62294213d38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:49.591737 22239 tablet_service.cc:1467] Processing CreateTablet for tablet a4379a5410d346489a49ce44fd10a21a (DEFAULT_TABLE table=test-table [id=e7816306f3c849d68311e62294213d38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:49.593225 21848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4379a5410d346489a49ce44fd10a21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:49.593225 22239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4379a5410d346489a49ce44fd10a21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:49.603345 22379 tablet_bootstrap.cc:492] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6: Bootstrap starting.
I20260808 02:48:49.606833 22379 tablet_bootstrap.cc:654] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:49.607625 21941 heartbeater.cc:503] Master 127.25.137.254:45533 was elected leader, sending a full tablet report...
I20260808 02:48:49.608014 22379 log.cc:824] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:49.613322 22381 tablet_bootstrap.cc:492] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: Bootstrap starting.
I20260808 02:48:49.617619 22381 tablet_bootstrap.cc:654] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:49.618083 22380 tablet_bootstrap.cc:492] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb: Bootstrap starting.
I20260808 02:48:49.618984 22381 log.cc:824] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:49.620098 22379 tablet_bootstrap.cc:492] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6: No bootstrap required, opened a new log
I20260808 02:48:49.620398 22379 ts_tablet_manager.cc:1397] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260808 02:48:49.621704 22381 tablet_bootstrap.cc:492] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: No bootstrap required, opened a new log
I20260808 02:48:49.621747 22380 tablet_bootstrap.cc:654] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:49.621981 22381 ts_tablet_manager.cc:1397] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 02:48:49.622965 22380 log.cc:824] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:49.625999 22380 tablet_bootstrap.cc:492] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb: No bootstrap required, opened a new log
I20260808 02:48:49.626251 22380 ts_tablet_manager.cc:1397] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260808 02:48:49.629897 22381 raft_consensus.cc:348] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34563 } } peers { permanent_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45811 } } peers { permanent_uuid: "fdcaef5ad46641ce97d298c881cddb30" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 45521 } }
I20260808 02:48:49.630391 22381 raft_consensus.cc:374] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:49.630497 22381 raft_consensus.cc:729] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdcaef5ad46641ce97d298c881cddb30, State: Initialized, Role: FOLLOWER
I20260808 02:48:49.630540 22379 raft_consensus.cc:348] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34563 } } peers { permanent_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45811 } } peers { permanent_uuid: "fdcaef5ad46641ce97d298c881cddb30" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 45521 } }
I20260808 02:48:49.630942 22379 raft_consensus.cc:374] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:49.631068 22379 raft_consensus.cc:729] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3500bff31aac4efeb4a1fd7a959ac7e6, State: Initialized, Role: FOLLOWER
I20260808 02:48:49.631137 22381 consensus_queue.cc:260] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34563 } } peers { permanent_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45811 } } peers { permanent_uuid: "fdcaef5ad46641ce97d298c881cddb30" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 45521 } }
I20260808 02:48:49.631636 22379 consensus_queue.cc:260] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34563 } } peers { permanent_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45811 } } peers { permanent_uuid: "fdcaef5ad46641ce97d298c881cddb30" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 45521 } }
I20260808 02:48:49.632478 22353 heartbeater.cc:503] Master 127.25.137.254:45533 was elected leader, sending a full tablet report...
I20260808 02:48:49.633185 22381 ts_tablet_manager.cc:1428] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260808 02:48:49.634238 22134 heartbeater.cc:503] Master 127.25.137.254:45533 was elected leader, sending a full tablet report...
I20260808 02:48:49.634842 22379 ts_tablet_manager.cc:1428] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6: Time spent starting tablet: real 0.014s user 0.004s sys 0.011s
I20260808 02:48:49.637473 22380 raft_consensus.cc:348] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34563 } } peers { permanent_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45811 } } peers { permanent_uuid: "fdcaef5ad46641ce97d298c881cddb30" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 45521 } }
I20260808 02:48:49.637931 22380 raft_consensus.cc:374] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:49.638051 22380 raft_consensus.cc:729] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f16d50d5a234bf08856d88e6bdf30cb, State: Initialized, Role: FOLLOWER
I20260808 02:48:49.638651 22380 consensus_queue.cc:260] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34563 } } peers { permanent_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45811 } } peers { permanent_uuid: "fdcaef5ad46641ce97d298c881cddb30" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 45521 } }
I20260808 02:48:49.639986 22380 ts_tablet_manager.cc:1428] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
I20260808 02:48:49.720714 22389 raft_consensus.cc:482] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:48:49.721050 22389 raft_consensus.cc:504] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34563 } } peers { permanent_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45811 } } peers { permanent_uuid: "fdcaef5ad46641ce97d298c881cddb30" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 45521 } }
I20260808 02:48:49.722760 22389 leader_election.cc:288] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3500bff31aac4efeb4a1fd7a959ac7e6 (127.25.137.194:34563), 4f16d50d5a234bf08856d88e6bdf30cb (127.25.137.193:45811)
I20260808 02:48:49.729264 22372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.723206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:55323 (local address 127.25.137.194:34563)
0808 02:48:49.723402 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:49.723409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:49.723632 (+ 223us) server_negotiation.cc:407] Connection header received
0808 02:48:49.723867 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:49.723873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:49.723970 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:49.724081 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:49.724658 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.725559 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.727412 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.728387 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.728500 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.728551 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:49.728863 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 02:48:49.728964 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:48:49.729338 22355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.723115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:34563 (local address 127.25.137.195:55323)
0808 02:48:49.723504 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:49.723528 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:49.723699 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:49.724184 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:49.724190 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:49.724225 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:49.724454 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.724468 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.725720 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.725725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.727240 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.727253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.728529 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.728535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.728706 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.728743 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:48:49.728844 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 02:48:49.728979 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268}
I20260808 02:48:49.729835 22373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.723199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:48703 (local address 127.25.137.193:45811)
0808 02:48:49.723403 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:49.723408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:49.724110 (+ 702us) server_negotiation.cc:407] Connection header received
0808 02:48:49.724311 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:49.724316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:49.724401 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:49.724508 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:49.725184 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.726050 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.727893 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.728873 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.729035 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.729101 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:49.729468 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 02:48:49.729583 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:48:49.730430 22394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.723184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.25.137.195:48703)
0808 02:48:49.723986 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:49.724008 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:49.724130 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:49.724545 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:49.724549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:49.724574 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:49.724951 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.724963 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.726209 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.726213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.727697 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:49.727712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.729054 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.729060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:49.729236 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.729269 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:48:49.730015 (+ 746us) client_negotiation.cc:240] Negotiation successful
0808 02:48:49.730143 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":406,"threads_started":1}
I20260808 02:48:49.730455 22070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4379a5410d346489a49ce44fd10a21a" candidate_uuid: "fdcaef5ad46641ce97d298c881cddb30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" is_pre_election: true
I20260808 02:48:49.731032 22070 raft_consensus.cc:2454] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdcaef5ad46641ce97d298c881cddb30 in term 0.
I20260808 02:48:49.731446 21877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4379a5410d346489a49ce44fd10a21a" candidate_uuid: "fdcaef5ad46641ce97d298c881cddb30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" is_pre_election: true
I20260808 02:48:49.731768 22164 leader_election.cc:302] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3500bff31aac4efeb4a1fd7a959ac7e6, fdcaef5ad46641ce97d298c881cddb30; no voters:
I20260808 02:48:49.731957 21877 raft_consensus.cc:2454] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdcaef5ad46641ce97d298c881cddb30 in term 0.
I20260808 02:48:49.732215 22389 raft_consensus.cc:2781] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:48:49.732368 22389 raft_consensus.cc:482] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:48:49.732489 22389 raft_consensus.cc:3037] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:49.734826 22389 raft_consensus.cc:504] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34563 } } peers { permanent_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45811 } } peers { permanent_uuid: "fdcaef5ad46641ce97d298c881cddb30" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 45521 } }
I20260808 02:48:49.735404 22389 leader_election.cc:288] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [CANDIDATE]: Term 1 election: Requested vote from peers 3500bff31aac4efeb4a1fd7a959ac7e6 (127.25.137.194:34563), 4f16d50d5a234bf08856d88e6bdf30cb (127.25.137.193:45811)
I20260808 02:48:49.735862 22070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4379a5410d346489a49ce44fd10a21a" candidate_uuid: "fdcaef5ad46641ce97d298c881cddb30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6"
I20260808 02:48:49.735992 21877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4379a5410d346489a49ce44fd10a21a" candidate_uuid: "fdcaef5ad46641ce97d298c881cddb30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f16d50d5a234bf08856d88e6bdf30cb"
I20260808 02:48:49.736063 22070 raft_consensus.cc:3037] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:49.736210 21877 raft_consensus.cc:3037] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:49.738476 22070 raft_consensus.cc:2454] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdcaef5ad46641ce97d298c881cddb30 in term 1.
I20260808 02:48:49.738679 21877 raft_consensus.cc:2454] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdcaef5ad46641ce97d298c881cddb30 in term 1.
I20260808 02:48:49.738897 22164 leader_election.cc:302] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3500bff31aac4efeb4a1fd7a959ac7e6, fdcaef5ad46641ce97d298c881cddb30; no voters:
I20260808 02:48:49.739252 22389 raft_consensus.cc:2781] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:48:49.739722 22389 raft_consensus.cc:686] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 1 LEADER]: Becoming Leader. State: Replica: fdcaef5ad46641ce97d298c881cddb30, State: Running, Role: LEADER
I20260808 02:48:49.740252 22389 consensus_queue.cc:237] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34563 } } peers { permanent_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45811 } } peers { permanent_uuid: "fdcaef5ad46641ce97d298c881cddb30" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 45521 } }
I20260808 02:48:49.744951 21657 catalog_manager.cc:5495] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdcaef5ad46641ce97d298c881cddb30 (127.25.137.195). New cstate: current_term: 1 leader_uuid: "fdcaef5ad46641ce97d298c881cddb30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdcaef5ad46641ce97d298c881cddb30" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 45521 } health_report { overall_health: HEALTHY } } }
W20260808 02:48:49.751423 22354 tablet.cc:2369] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:48:49.769405 22378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.763388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40642 (local address 127.25.137.195:45521)
0808 02:48:49.763574 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:49.763582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:49.763686 (+ 104us) server_negotiation.cc:407] Connection header received
0808 02:48:49.763934 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:49.763938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:49.764032 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:49.764170 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:48:49.764600 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.765413 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.767131 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.767581 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.767696 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.768264 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:48:49.768776 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:48:49.768917 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:49.769055 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 02:48:49.769142 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:48:49.777434 22070 raft_consensus.cc:1261] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6 [term 1 FOLLOWER]: Refusing update from remote peer fdcaef5ad46641ce97d298c881cddb30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:48:49.777463 21877 raft_consensus.cc:1261] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb [term 1 FOLLOWER]: Refusing update from remote peer fdcaef5ad46641ce97d298c881cddb30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:48:49.778285 22395 consensus_queue.cc:1035] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3500bff31aac4efeb4a1fd7a959ac7e6" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 34563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:48:49.778638 22389 consensus_queue.cc:1035] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f16d50d5a234bf08856d88e6bdf30cb" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:48:49.789070 22400 mvcc.cc:205] Tried to move back new op lower bound from 7316100422759038976 to 7316100422620389376. Current Snapshot: MvccSnapshot[applied={T|T < 7316100422759038976}]
I20260808 02:48:49.802870 21657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:48:49.803158 21657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:48:49.806991 21657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4841ea0bede44e51aa96b56a0fb5d57a: Sending DeleteTablet for 3 replicas of tablet a4379a5410d346489a49ce44fd10a21a
I20260808 02:48:49.808174 21848 tablet_service.cc:1514] Processing DeleteTablet for tablet a4379a5410d346489a49ce44fd10a21a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:49 UTC) from {username='slave'} at 127.0.0.1:53742
I20260808 02:48:49.808180 22043 tablet_service.cc:1514] Processing DeleteTablet for tablet a4379a5410d346489a49ce44fd10a21a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:49 UTC) from {username='slave'} at 127.0.0.1:46042
I20260808 02:48:49.808277 22239 tablet_service.cc:1514] Processing DeleteTablet for tablet a4379a5410d346489a49ce44fd10a21a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:49 UTC) from {username='slave'} at 127.0.0.1:40626
I20260808 02:48:49.809692 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 21754
I20260808 02:48:49.820927 22413 tablet_replica.cc:330] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6: stopping tablet replica
I20260808 02:48:49.821661 22413 raft_consensus.cc:2229] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:48:49.822363 22413 raft_consensus.cc:2258] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:49.824718 22413 ts_tablet_manager.cc:1905] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:49.830528 22373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.810897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53746 (local address 127.25.137.193:45811)
0808 02:48:49.811082 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:49.811089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:49.821257 (+ 10168us) server_negotiation.cc:407] Connection header received
0808 02:48:49.821801 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:49.821808 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:49.821948 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:49.822038 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:49.822803 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.823957 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.826077 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.826800 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.826905 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:49.828530 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:49.828543 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:49.828560 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:49.829427 (+ 867us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:49.829517 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:49.829545 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:49.829636 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:49.829991 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 02:48:49.830124 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:48:49.831512 22411 tablet_replica.cc:330] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb: stopping tablet replica
I20260808 02:48:49.831734 21936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:49.833249 22413 ts_tablet_manager.cc:1918] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:49.833431 22413 log.cc:1196] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/a4379a5410d346489a49ce44fd10a21a
I20260808 02:48:49.834136 22413 ts_tablet_manager.cc:1939] T a4379a5410d346489a49ce44fd10a21a P 3500bff31aac4efeb4a1fd7a959ac7e6: Deleting consensus metadata
I20260808 02:48:49.836325 21633 catalog_manager.cc:4841] TS 3500bff31aac4efeb4a1fd7a959ac7e6 (127.25.137.194:34563): tablet a4379a5410d346489a49ce44fd10a21a (table test-table [id=e7816306f3c849d68311e62294213d38]) successfully deleted
I20260808 02:48:49.838009 22412 tablet_replica.cc:330] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: stopping tablet replica
I20260808 02:48:49.838512 22412 raft_consensus.cc:2229] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:48:49.839311 22412 raft_consensus.cc:2258] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:49.841462 22412 ts_tablet_manager.cc:1905] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:49.847250 22412 ts_tablet_manager.cc:1918] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:49.847416 22412 log.cc:1196] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/a4379a5410d346489a49ce44fd10a21a
I20260808 02:48:49.847898 22412 ts_tablet_manager.cc:1939] T a4379a5410d346489a49ce44fd10a21a P fdcaef5ad46641ce97d298c881cddb30: Deleting consensus metadata
I20260808 02:48:49.849531 21632 catalog_manager.cc:4841] TS fdcaef5ad46641ce97d298c881cddb30 (127.25.137.195:45521): tablet a4379a5410d346489a49ce44fd10a21a (table test-table [id=e7816306f3c849d68311e62294213d38]) successfully deleted
I20260808 02:48:49.832011 22411 raft_consensus.cc:2229] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:48:49.951467 22411 raft_consensus.cc:2258] T a4379a5410d346489a49ce44fd10a21a P 4f16d50d5a234bf08856d88e6bdf30cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:49.951741 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 21754
W20260808 02:48:49.979444 21637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.137.193:45811 (error 108)
I20260808 02:48:49.979722 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 21948
I20260808 02:48:49.981926 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.981453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53750)
0808 02:48:49.981615 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:49.981752 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:48:49.982249 21637 catalog_manager.cc:4584] TS 4f16d50d5a234bf08856d88e6bdf30cb (127.25.137.193:45811): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a4379a5410d346489a49ce44fd10a21a: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
I20260808 02:48:50.035794 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:50.035298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53766)
0808 02:48:50.035535 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:50.035628 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 02:48:50.036811 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:50.036462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53780)
0808 02:48:50.036613 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:50.036685 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 02:48:50.097360 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:50.096832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53792)
0808 02:48:50.097091 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:50.097189 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 02:48:50.098346 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:50.098012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53796)
0808 02:48:50.098163 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:50.098233 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 02:48:50.178982 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:50.178480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53804)
0808 02:48:50.178695 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:50.178787 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 02:48:50.180047 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:50.179635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53816)
0808 02:48:50.179839 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:50.179912 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260808 02:48:50.352808 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:50.352254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53820)
0808 02:48:50.352478 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:50.352584 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 02:48:50.353853 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:50.353505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53830)
0808 02:48:50.353642 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:50.353727 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 02:48:50.647001 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:50.646489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53844)
0808 02:48:50.646701 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:50.646809 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 02:48:50.648003 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:50.647676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45811 (local address 127.0.0.1:53850)
0808 02:48:50.647800 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:50.647878 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 02:48:50.847653 22415 sasl_common.cc:323] Time spent creating new SASL kudu client: real 0.862s user 0.000s sys 0.000s
I20260808 02:48:50.849768 22372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:49.981045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46046 (local address 127.25.137.194:34563)
0808 02:48:49.981246 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:49.981253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:49.981276 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:48:49.981350 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:49.981356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:49.981492 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:49.981608 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:49.982162 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.983066 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.984436 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:49.984874 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:49.984985 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:50.848216 (+863231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:50.848228 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:50.848245 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:50.848889 (+ 644us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:50.848948 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:50.848974 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:50.849152 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:50.849315 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 02:48:50.849452 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:48:50.850579 22128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:50.938550 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 21948
I20260808 02:48:50.962090 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 22138
I20260808 02:48:50.968881 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:50.962836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40650 (local address 127.25.137.195:45521)
0808 02:48:50.963309 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:50.963313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:50.963330 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:48:50.963388 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:50.963392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:50.963474 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:50.963561 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:50.963970 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:50.964726 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:50.966002 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:50.966418 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:50.966512 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:50.967518 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:50.967528 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:50.967544 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:50.968139 (+ 595us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:50.968184 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:50.968206 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:50.968307 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:50.968423 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 02:48:50.968500 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":247,"threads_started":1}
I20260808 02:48:50.969663 22344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:51.084131 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 22138
I20260808 02:48:51.111538 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 21607
I20260808 02:48:51.112818 21711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:51.222312 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 21607
2026-08-08T02:48:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3745 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:48:51 dist-test-slave-k5rr krb5kdc[22429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:48:51 dist-test-slave-k5rr krb5kdc[22429](info): set up 2 sockets
Aug 08 02:48:51 dist-test-slave-k5rr krb5kdc[22429](info): commencing operation
krb5kdc: starting...
W20260808 02:48:52.471400 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.006s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:48:52 dist-test-slave-k5rr krb5kdc[22429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:48:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:48:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:48:52.789350 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45129
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:45129
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:53.126994 22448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:53.127288 22448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:53.127383 22448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:53.134898 22448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:48:53.135018 22448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:53.135073 22448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:53.135133 22448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:48:53.135175 22448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:48:53.145061 22448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:45129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:53.146312 22448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:53.147549 22448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:53.159119 22453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:53.159049 22457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:53.159067 22454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:53.160006 22448 server_base.cc:1034] running on GCE node
I20260808 02:48:53.160764 22448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:53.162186 22448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:53.163390 22448 hybrid_clock.cc:648] HybridClock initialized: now 1786157333163329 us; error 50 us; skew 500 ppm
I20260808 02:48:53.163751 22448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:53 dist-test-slave-k5rr krb5kdc[22429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157333, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:53.166883 22448 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:48:53.168394 22448 webserver.cc:460] Webserver started at http://127.25.137.254:45243/ using document root <none> and password file <none>
I20260808 02:48:53.169070 22448 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:53.169179 22448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:53.169445 22448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:53.172454 22448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "d3816db59d97485aac548563c59ab2c2"
format_stamp: "Formatted at 2026-08-08 02:48:53 on dist-test-slave-k5rr"
server_key: "7b1d096f7834ab08bc39eaeaa2b90096"
server_key_iv: "e264159953ed8bcc6cef865ac69347fe"
server_key_version: "encryptionkey@0"
I20260808 02:48:53.173276 22448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "d3816db59d97485aac548563c59ab2c2"
format_stamp: "Formatted at 2026-08-08 02:48:53 on dist-test-slave-k5rr"
server_key: "7b1d096f7834ab08bc39eaeaa2b90096"
server_key_iv: "e264159953ed8bcc6cef865ac69347fe"
server_key_version: "encryptionkey@0"
I20260808 02:48:53.178413 22448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 02:48:53.181725 22469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:53.183061 22448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:48:53.183207 22448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "d3816db59d97485aac548563c59ab2c2"
format_stamp: "Formatted at 2026-08-08 02:48:53 on dist-test-slave-k5rr"
server_key: "7b1d096f7834ab08bc39eaeaa2b90096"
server_key_iv: "e264159953ed8bcc6cef865ac69347fe"
server_key_version: "encryptionkey@0"
I20260808 02:48:53.183370 22448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:53.200162 22448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:53.203841 22448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:53.204066 22448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:53.221324 22448 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:45129
I20260808 02:48:53.221333 22546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:45129 every 8 connection(s)
I20260808 02:48:53.222618 22448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:48:53.226229 22547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:53.228600 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 22448
I20260808 02:48:53.228786 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:48:53.229369 26151 external_mini_cluster.cc:1428] Setting key 51372345521e81229613c0c088932abc
I20260808 02:48:53.238407 22547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2: Bootstrap starting.
I20260808 02:48:53.242158 22547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:53.243484 22547 log.cc:824] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:48:53 dist-test-slave-k5rr krb5kdc[22429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:48:53.246634 22547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2: No bootstrap required, opened a new log
I20260808 02:48:53.255223 22551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:53.231384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47738 (local address 127.25.137.254:45129)
0808 02:48:53.232291 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:53.232303 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:53.232346 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:48:53.234166 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:53.234189 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:53.234594 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:53.234942 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:53.236909 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:53.238292 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:53.240182 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:53.240914 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:53.241100 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:53.245956 (+ 4856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:53.245967 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:53.245986 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:53.246033 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:53.250018 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:53.250706 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:53.250715 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:53.250723 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:53.250949 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:53.251410 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:53.251415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:53.251418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:53.252181 (+ 763us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:53.252490 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:53.253278 (+ 788us) server_negotiation.cc:299] Negotiation successful
0808 02:48:53.253759 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":386,"threads_started":1}
I20260808 02:48:53.257349 22547 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3816db59d97485aac548563c59ab2c2" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45129 } }
I20260808 02:48:53.257715 22547 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:53.257831 22547 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3816db59d97485aac548563c59ab2c2, State: Initialized, Role: FOLLOWER
I20260808 02:48:53.258443 22547 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3816db59d97485aac548563c59ab2c2" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45129 } }
I20260808 02:48:53.258701 22547 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:48:53.258836 22547 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:48:53.258987 22547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:53.261252 22547 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3816db59d97485aac548563c59ab2c2" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45129 } }
I20260808 02:48:53.261726 22547 leader_election.cc:302] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3816db59d97485aac548563c59ab2c2; no voters:
I20260808 02:48:53.262365 22547 leader_election.cc:288] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:48:53.262509 22554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:48:53.263213 22554 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [term 1 LEADER]: Becoming Leader. State: Replica: d3816db59d97485aac548563c59ab2c2, State: Running, Role: LEADER
I20260808 02:48:53.263787 22554 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3816db59d97485aac548563c59ab2c2" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45129 } }
I20260808 02:48:53.264575 22547 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:48:53.266739 22555 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3816db59d97485aac548563c59ab2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3816db59d97485aac548563c59ab2c2" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45129 } } }
I20260808 02:48:53.266798 22557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3816db59d97485aac548563c59ab2c2. Latest consensus state: current_term: 1 leader_uuid: "d3816db59d97485aac548563c59ab2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3816db59d97485aac548563c59ab2c2" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45129 } } }
I20260808 02:48:53.267618 22555 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:53.267745 22557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:53.269981 22567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:48:53.274768 22567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:48:53.282725 22567 catalog_manager.cc:1347] Generated new cluster ID: a0f9e2860ce74a8ba9898c7b5503edea
I20260808 02:48:53.282851 22567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:48:53.300756 22567 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:48:53.302785 22567 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:48:53.311287 22567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2: Generated new TSK 0
I20260808 02:48:53.312400 22567 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:48:53.386366 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45129
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:53.735847 22584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:53.736132 22584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:53.736270 22584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:53.743659 22584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:53.743784 22584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:53.743943 22584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:48:53.754004 22584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:53.755303 22584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:53.756623 22584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:53.769601 22597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:53.769690 22593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:53.769619 22594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:53.770787 22584 server_base.cc:1034] running on GCE node
I20260808 02:48:53.771396 22584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:53.772324 22584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:53.773536 22584 hybrid_clock.cc:648] HybridClock initialized: now 1786157333773475 us; error 45 us; skew 500 ppm
I20260808 02:48:53.773968 22584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:53 dist-test-slave-k5rr krb5kdc[22429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157333, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:53.777284 22584 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:48:53.778688 22584 webserver.cc:460] Webserver started at http://127.25.137.193:41877/ using document root <none> and password file <none>
I20260808 02:48:53.779348 22584 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:53.779477 22584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:53.779804 22584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:53.782792 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "ffc1e159ee174b918022040176b00401"
format_stamp: "Formatted at 2026-08-08 02:48:53 on dist-test-slave-k5rr"
server_key: "aeac1121321ee721f591f8ad0fc8392f"
server_key_iv: "dd126503df08249ff5828e2cee84a4f3"
server_key_version: "encryptionkey@0"
I20260808 02:48:53.783576 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "ffc1e159ee174b918022040176b00401"
format_stamp: "Formatted at 2026-08-08 02:48:53 on dist-test-slave-k5rr"
server_key: "aeac1121321ee721f591f8ad0fc8392f"
server_key_iv: "dd126503df08249ff5828e2cee84a4f3"
server_key_version: "encryptionkey@0"
I20260808 02:48:53.788997 22584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:48:53.792585 22610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:53.793969 22584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:48:53.794135 22584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "ffc1e159ee174b918022040176b00401"
format_stamp: "Formatted at 2026-08-08 02:48:53 on dist-test-slave-k5rr"
server_key: "aeac1121321ee721f591f8ad0fc8392f"
server_key_iv: "dd126503df08249ff5828e2cee84a4f3"
server_key_version: "encryptionkey@0"
I20260808 02:48:53.794332 22584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:53.805862 22584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:53.809670 22584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:53.809967 22584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:53.811004 22584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:53.812461 22584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:53.812549 22584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:53.812675 22584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:53.812743 22584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:53.845448 22584 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:43601
I20260808 02:48:53.845510 22797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:43601 every 8 connection(s)
I20260808 02:48:53.846824 22584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:48:53.855271 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 22584
I20260808 02:48:53.855449 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:48:53.855911 26151 external_mini_cluster.cc:1428] Setting key 84863b0b1834cd0bdfbbd28725e21305
Aug 08 02:48:53 dist-test-slave-k5rr krb5kdc[22429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157333, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:48:53.867192 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:53.849479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:36921 (local address 127.25.137.254:45129)
0808 02:48:53.849943 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:53.849948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:53.850877 (+ 929us) server_negotiation.cc:407] Connection header received
0808 02:48:53.852681 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:53.852687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:53.852773 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:53.852873 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:53.854355 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:53.855204 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:53.857026 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:53.857469 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:53.857589 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:53.861182 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:53.861188 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:53.861192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:53.861227 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:53.863989 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:53.864781 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:53.864787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:53.864790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:53.864933 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:53.865426 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:53.865430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:53.865433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:53.865791 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:53.865925 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:53.866662 (+ 737us) server_negotiation.cc:299] Negotiation successful
0808 02:48:53.866875 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":218,"threads_started":1}
I20260808 02:48:53.868000 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:53.850027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45129 (local address 127.25.137.193:36921)
0808 02:48:53.850721 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:53.850761 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:53.852426 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:53.853133 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:53.853141 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:53.853650 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:53.854182 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:53.854199 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:53.855387 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:53.855393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:53.856854 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:53.856881 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:53.857706 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:53.857714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:53.857978 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:53.858611 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:48:53.858631 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:53.860950 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:53.864142 (+ 3192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:53.864149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:53.864160 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:53.864660 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:53.865069 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:53.865073 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:53.865075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:53.865299 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:53.865916 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:53.865921 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:53.866269 (+ 348us) client_negotiation.cc:769] Sending connection context
0808 02:48:53.866511 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 02:48:53.866888 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":273,"threads_started":1}
I20260808 02:48:53.869539 22798 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45129
I20260808 02:48:53.869864 22798 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:53.870649 22798 heartbeater.cc:511] Master 127.25.137.254:45129 requested a full tablet report, sending...
I20260808 02:48:53.873131 22493 ts_manager.cc:194] Registered new tserver with Master: ffc1e159ee174b918022040176b00401 (127.25.137.193:43601)
I20260808 02:48:53.875231 22493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:36921
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:48:53.926586 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45129
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:54.256107 22809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:54.256389 22809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:54.256513 22809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:54.263790 22809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:54.263898 22809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:54.264047 22809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:48:54.274035 22809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:54.275434 22809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:54.276774 22809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:54.290118 22818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:54.290124 22823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:54.290141 22819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:54.290879 22809 server_base.cc:1034] running on GCE node
I20260808 02:48:54.291597 22809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:54.292536 22809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:54.293730 22809 hybrid_clock.cc:648] HybridClock initialized: now 1786157334293663 us; error 61 us; skew 500 ppm
I20260808 02:48:54.294045 22809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:54 dist-test-slave-k5rr krb5kdc[22429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157334, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:54.297176 22809 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:48:54.298727 22809 webserver.cc:460] Webserver started at http://127.25.137.194:37325/ using document root <none> and password file <none>
I20260808 02:48:54.299443 22809 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:54.299573 22809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:54.299880 22809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:54.302918 22809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "ea2e58e0529b49859a928a494131febf"
format_stamp: "Formatted at 2026-08-08 02:48:54 on dist-test-slave-k5rr"
server_key: "afce9a2147c6994393da8fa5b59e4eef"
server_key_iv: "17c22036e12dee5bfba24b166b7b0dc5"
server_key_version: "encryptionkey@0"
I20260808 02:48:54.303691 22809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "ea2e58e0529b49859a928a494131febf"
format_stamp: "Formatted at 2026-08-08 02:48:54 on dist-test-slave-k5rr"
server_key: "afce9a2147c6994393da8fa5b59e4eef"
server_key_iv: "17c22036e12dee5bfba24b166b7b0dc5"
server_key_version: "encryptionkey@0"
I20260808 02:48:54.308833 22809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 02:48:54.312463 22833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:54.313796 22809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:48:54.313941 22809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "ea2e58e0529b49859a928a494131febf"
format_stamp: "Formatted at 2026-08-08 02:48:54 on dist-test-slave-k5rr"
server_key: "afce9a2147c6994393da8fa5b59e4eef"
server_key_iv: "17c22036e12dee5bfba24b166b7b0dc5"
server_key_version: "encryptionkey@0"
I20260808 02:48:54.314126 22809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:54.332710 22809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:54.336603 22809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:54.336869 22809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:54.337908 22809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:54.339365 22809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:54.339442 22809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:54.339553 22809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:54.339607 22809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:54.370816 22809 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:35121
I20260808 02:48:54.370872 23006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:35121 every 8 connection(s)
I20260808 02:48:54.372066 22809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:48:54.376842 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 22809
I20260808 02:48:54.377038 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:48:54.377470 26151 external_mini_cluster.cc:1428] Setting key 85e4b00b6decb369b9f0a58f9fb464c5
Aug 08 02:48:54 dist-test-slave-k5rr krb5kdc[22429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157334, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:48:54.392342 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:54.374493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:34565 (local address 127.25.137.254:45129)
0808 02:48:54.375038 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:54.375044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:54.376138 (+ 1094us) server_negotiation.cc:407] Connection header received
0808 02:48:54.378497 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:54.378502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:54.378589 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:54.378688 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:54.380338 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.381077 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.382577 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.382990 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.383098 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:54.386562 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:54.386567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:54.386570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:54.386602 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:54.389279 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:54.389984 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:54.389990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:54.389992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:54.390137 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:54.390578 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:54.390582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:54.390584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:54.390863 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:54.391009 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:54.391724 (+ 715us) server_negotiation.cc:299] Negotiation successful
0808 02:48:54.391968 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":207,"threads_started":1}
I20260808 02:48:54.393348 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:54.375102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45129 (local address 127.25.137.194:34565)
0808 02:48:54.375949 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:54.375992 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:54.378189 (+ 2197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:54.378944 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:54.378952 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:54.379603 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:54.380168 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:54.380185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.381233 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.381237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:54.382441 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:54.382453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.383202 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.383210 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:54.383430 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:54.384023 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:48:54.384050 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:54.386351 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:54.389420 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:54.389429 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:54.389446 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:54.389869 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:54.390261 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:54.390266 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:54.390271 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:54.390468 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:54.390987 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:54.390994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:54.391358 (+ 364us) client_negotiation.cc:769] Sending connection context
0808 02:48:54.391617 (+ 259us) client_negotiation.cc:240] Negotiation successful
0808 02:48:54.392137 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":322,"threads_started":1}
I20260808 02:48:54.394776 23007 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45129
I20260808 02:48:54.395108 23007 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:54.396000 23007 heartbeater.cc:511] Master 127.25.137.254:45129 requested a full tablet report, sending...
I20260808 02:48:54.397645 22493 ts_manager.cc:194] Registered new tserver with Master: ea2e58e0529b49859a928a494131febf (127.25.137.194:35121)
I20260808 02:48:54.398705 22493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:34565
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:48:54.446408 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45129
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:48:54.779970 23015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:54.780401 23015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:54.780592 23015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:54.787835 23015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:54.787940 23015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:54.788092 23015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:48:54.797583 23015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:45129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:54.798873 23015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:54.800205 23015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:54.813153 23026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:54.813063 23028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:54.813093 23030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:54.814020 23015 server_base.cc:1034] running on GCE node
I20260808 02:48:54.814599 23015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:54.815567 23015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:54.816768 23015 hybrid_clock.cc:648] HybridClock initialized: now 1786157334816701 us; error 51 us; skew 500 ppm
I20260808 02:48:54.817104 23015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:48:54 dist-test-slave-k5rr krb5kdc[22429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157334, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:48:54.820178 23015 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:48:54.821609 23015 webserver.cc:460] Webserver started at http://127.25.137.195:42825/ using document root <none> and password file <none>
I20260808 02:48:54.822257 23015 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:54.822384 23015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:54.822683 23015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:54.825661 23015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "b645b415e3f5498798bbe9c98a6ba1a9"
format_stamp: "Formatted at 2026-08-08 02:48:54 on dist-test-slave-k5rr"
server_key: "28f59d6193dff68f929806714a8ad847"
server_key_iv: "12aa6863d27eb7bc63e57fbc4f13150c"
server_key_version: "encryptionkey@0"
I20260808 02:48:54.826427 23015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "b645b415e3f5498798bbe9c98a6ba1a9"
format_stamp: "Formatted at 2026-08-08 02:48:54 on dist-test-slave-k5rr"
server_key: "28f59d6193dff68f929806714a8ad847"
server_key_iv: "12aa6863d27eb7bc63e57fbc4f13150c"
server_key_version: "encryptionkey@0"
I20260808 02:48:54.831460 23015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:48:54.834887 23037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:54.836169 23015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 02:48:54.836338 23015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "b645b415e3f5498798bbe9c98a6ba1a9"
format_stamp: "Formatted at 2026-08-08 02:48:54 on dist-test-slave-k5rr"
server_key: "28f59d6193dff68f929806714a8ad847"
server_key_iv: "12aa6863d27eb7bc63e57fbc4f13150c"
server_key_version: "encryptionkey@0"
I20260808 02:48:54.836517 23015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:54.854184 23015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:54.858209 23015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:54.858474 23015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:54.859400 23015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:54.860677 23015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:54.860762 23015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:54.860882 23015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:54.860949 23015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:54.878384 22798 heartbeater.cc:503] Master 127.25.137.254:45129 was elected leader, sending a full tablet report...
I20260808 02:48:54.891731 23015 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:42025
I20260808 02:48:54.891764 23214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:42025 every 8 connection(s)
I20260808 02:48:54.892937 23015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:48:54.896432 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 23015
I20260808 02:48:54.896596 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:48:54.897140 26151 external_mini_cluster.cc:1428] Setting key 02dfb74bb9f5dca5b8b22c5b60a0f26d
Aug 08 02:48:54 dist-test-slave-k5rr krb5kdc[22429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157334, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:48:54.912137 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:54.895326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:50583 (local address 127.25.137.254:45129)
0808 02:48:54.895720 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:54.895725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:54.896808 (+ 1083us) server_negotiation.cc:407] Connection header received
0808 02:48:54.899112 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:54.899117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:54.899207 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:54.899331 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:54.900883 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.901691 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.903118 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.903503 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.903588 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:54.906718 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:54.906723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:54.906726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:54.906758 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:54.909215 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:54.909884 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:54.909890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:54.909894 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:54.910017 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:54.910420 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:54.910425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:54.910428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:54.910697 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:54.910848 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:54.911532 (+ 684us) server_negotiation.cc:299] Negotiation successful
0808 02:48:54.911757 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":185,"threads_started":1}
I20260808 02:48:54.913036 23220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:54.895872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:45129 (local address 127.25.137.195:50583)
0808 02:48:54.896653 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:54.896698 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:54.898806 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:54.899532 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:54.899540 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:54.900202 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:54.900736 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:54.900752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.901814 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.901817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:54.903013 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:54.903025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.903664 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.903669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:54.903856 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:54.904432 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:48:54.904458 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:54.906525 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:54.909347 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:54.909355 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:54.909370 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:54.909761 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:54.910120 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:48:54.910125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:48:54.910129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:48:54.910314 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:48:54.910839 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:54.910847 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:54.911159 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 02:48:54.911399 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 02:48:54.911921 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":318,"threads_started":1}
I20260808 02:48:54.914187 23215 heartbeater.cc:347] Connected to a master server at 127.25.137.254:45129
I20260808 02:48:54.914463 23215 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:54.915159 23215 heartbeater.cc:511] Master 127.25.137.254:45129 requested a full tablet report, sending...
I20260808 02:48:54.916543 22493 ts_manager.cc:194] Registered new tserver with Master: b645b415e3f5498798bbe9c98a6ba1a9 (127.25.137.195:42025)
I20260808 02:48:54.917364 22493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:50583
I20260808 02:48:54.924155 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:48:54.940021 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:54.928128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47744 (local address 127.25.137.254:45129)
0808 02:48:54.928279 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:54.928283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:54.928718 (+ 435us) server_negotiation.cc:407] Connection header received
0808 02:48:54.928868 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:54.928872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:54.928939 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:54.929051 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:54.930364 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.931049 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.932293 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.932673 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.932756 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:54.935329 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:54.935334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:54.935337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:54.935367 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:54.937650 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:54.938387 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:54.938390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:54.938393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:54.938498 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:54.938816 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:54.938819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:54.938821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:54.939106 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:54.939305 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:54.939647 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 02:48:54.939780 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:48:54.943701 22493 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:48:54.950474 22493 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:48:54.986562 23238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:54.963246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51610 (local address 127.25.137.194:35121)
0808 02:48:54.977422 (+ 14176us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:54.977431 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:54.977461 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:48:54.979327 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:54.979335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:54.979513 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:54.979758 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:54.980602 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.981978 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.983945 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.984990 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.985179 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:54.985377 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:54.986050 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 02:48:54.986224 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14033,"thread_start_us":13919,"threads_started":1}
I20260808 02:48:54.986589 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:54.962418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:42025 (local address 127.0.0.1:52256)
0808 02:48:54.975569 (+ 13151us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:54.975630 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:54.976814 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:54.978668 (+ 1854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:54.978675 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:54.978711 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:54.979149 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:54.979163 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.981127 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.981133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:54.983645 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:54.983664 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.985405 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.985412 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:54.985674 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:54.985733 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 02:48:54.985860 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 02:48:54.986105 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12791,"mutex_wait_us":49,"thread_start_us":14608,"threads_started":1}
I20260808 02:48:54.987021 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:54.963134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:35121 (local address 127.0.0.1:51610)
0808 02:48:54.976816 (+ 13682us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:54.976838 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:54.976976 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:54.979992 (+ 3016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:54.979996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:54.980016 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:54.980432 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:54.980445 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.982152 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.982156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:54.983790 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:54.983806 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.985534 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.985541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:54.985708 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:54.985743 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:48:54.985875 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 02:48:54.986490 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13538,"mutex_wait_us":423,"thread_start_us":12737,"threads_started":1}
I20260808 02:48:54.987255 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:54.977801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47882 (local address 127.25.137.193:43601)
0808 02:48:54.979170 (+ 1369us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:54.979178 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:54.979204 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:48:54.979296 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:54.979303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:54.979518 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:54.979725 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:54.980400 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.981734 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.984217 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.985639 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.985825 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:54.985985 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:54.986697 (+ 712us) server_negotiation.cc:299] Negotiation successful
0808 02:48:54.986853 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1214,"thread_start_us":1056,"threads_started":1}
I20260808 02:48:54.987700 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:54.977673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:43601 (local address 127.0.0.1:47882)
0808 02:48:54.978246 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:54.978270 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:54.978389 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:54.979751 (+ 1362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:54.979757 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:54.979784 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:54.980198 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:54.980212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.981912 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.981918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:54.984055 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:54.984069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.986135 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.986140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:54.986312 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:54.986342 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 02:48:54.986445 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 02:48:54.986573 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":342,"threads_started":1}
I20260808 02:48:54.988648 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:54.962661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52256 (local address 127.25.137.195:42025)
0808 02:48:54.978002 (+ 15341us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:54.978011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:54.978042 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:48:54.978148 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:54.978155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:54.978431 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:54.979334 (+ 903us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:54.979549 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.980986 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.983826 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:54.985191 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:54.987184 (+ 1993us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:54.987348 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:54.987423 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 02:48:54.987576 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15124,"thread_start_us":15819,"threads_started":1}
I20260808 02:48:54.991240 22694 tablet_service.cc:1467] Processing CreateTablet for tablet 3f1bdb73bd584c68961f28b24cfe9b66 (DEFAULT_TABLE table=test-table [id=5d54a01f19f24070bc62c84924fdd47d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:54.991240 22909 tablet_service.cc:1467] Processing CreateTablet for tablet 3f1bdb73bd584c68961f28b24cfe9b66 (DEFAULT_TABLE table=test-table [id=5d54a01f19f24070bc62c84924fdd47d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:54.991798 23115 tablet_service.cc:1467] Processing CreateTablet for tablet 3f1bdb73bd584c68961f28b24cfe9b66 (DEFAULT_TABLE table=test-table [id=5d54a01f19f24070bc62c84924fdd47d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:54.992731 22694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f1bdb73bd584c68961f28b24cfe9b66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:54.992731 22909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f1bdb73bd584c68961f28b24cfe9b66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:54.992960 23115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f1bdb73bd584c68961f28b24cfe9b66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:55.004185 23241 tablet_bootstrap.cc:492] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: Bootstrap starting.
I20260808 02:48:55.008231 23242 tablet_bootstrap.cc:492] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: Bootstrap starting.
I20260808 02:48:55.009557 23241 tablet_bootstrap.cc:654] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:55.010883 23241 log.cc:824] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:55.011834 23242 tablet_bootstrap.cc:654] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:55.011853 23243 tablet_bootstrap.cc:492] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: Bootstrap starting.
I20260808 02:48:55.013108 23242 log.cc:824] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:55.013826 23241 tablet_bootstrap.cc:492] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: No bootstrap required, opened a new log
I20260808 02:48:55.014114 23241 ts_tablet_manager.cc:1397] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 02:48:55.015218 23243 tablet_bootstrap.cc:654] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:55.016163 23242 tablet_bootstrap.cc:492] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: No bootstrap required, opened a new log
I20260808 02:48:55.016428 23242 ts_tablet_manager.cc:1397] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 02:48:55.016431 23243 log.cc:824] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:55.022398 23241 raft_consensus.cc:348] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 42025 } } peers { permanent_uuid: "ffc1e159ee174b918022040176b00401" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43601 } } peers { permanent_uuid: "ea2e58e0529b49859a928a494131febf" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 35121 } }
I20260808 02:48:55.022760 23241 raft_consensus.cc:374] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:55.022876 23241 raft_consensus.cc:729] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b645b415e3f5498798bbe9c98a6ba1a9, State: Initialized, Role: FOLLOWER
I20260808 02:48:55.023464 23241 consensus_queue.cc:260] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 42025 } } peers { permanent_uuid: "ffc1e159ee174b918022040176b00401" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43601 } } peers { permanent_uuid: "ea2e58e0529b49859a928a494131febf" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 35121 } }
I20260808 02:48:55.023638 23243 tablet_bootstrap.cc:492] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: No bootstrap required, opened a new log
I20260808 02:48:55.023892 23243 ts_tablet_manager.cc:1397] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.009s
I20260808 02:48:55.024487 23215 heartbeater.cc:503] Master 127.25.137.254:45129 was elected leader, sending a full tablet report...
I20260808 02:48:55.025032 23241 ts_tablet_manager.cc:1428] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260808 02:48:55.026971 23242 raft_consensus.cc:348] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 42025 } } peers { permanent_uuid: "ffc1e159ee174b918022040176b00401" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43601 } } peers { permanent_uuid: "ea2e58e0529b49859a928a494131febf" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 35121 } }
I20260808 02:48:55.027330 23242 raft_consensus.cc:374] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:55.027427 23242 raft_consensus.cc:729] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2e58e0529b49859a928a494131febf, State: Initialized, Role: FOLLOWER
I20260808 02:48:55.028004 23242 consensus_queue.cc:260] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 42025 } } peers { permanent_uuid: "ffc1e159ee174b918022040176b00401" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43601 } } peers { permanent_uuid: "ea2e58e0529b49859a928a494131febf" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 35121 } }
I20260808 02:48:55.029549 23242 ts_tablet_manager.cc:1428] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: Time spent starting tablet: real 0.013s user 0.007s sys 0.007s
I20260808 02:48:55.029794 23007 heartbeater.cc:503] Master 127.25.137.254:45129 was elected leader, sending a full tablet report...
I20260808 02:48:55.034416 23243 raft_consensus.cc:348] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 42025 } } peers { permanent_uuid: "ffc1e159ee174b918022040176b00401" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43601 } } peers { permanent_uuid: "ea2e58e0529b49859a928a494131febf" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 35121 } }
I20260808 02:48:55.034740 23243 raft_consensus.cc:374] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:55.034821 23243 raft_consensus.cc:729] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc1e159ee174b918022040176b00401, State: Initialized, Role: FOLLOWER
I20260808 02:48:55.035269 23243 consensus_queue.cc:260] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 42025 } } peers { permanent_uuid: "ffc1e159ee174b918022040176b00401" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43601 } } peers { permanent_uuid: "ea2e58e0529b49859a928a494131febf" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 35121 } }
I20260808 02:48:55.036515 23243 ts_tablet_manager.cc:1428] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: Time spent starting tablet: real 0.012s user 0.003s sys 0.007s
W20260808 02:48:55.099164 22801 tablet.cc:2369] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:48:55.124037 23008 tablet.cc:2369] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:48:55.144506 23216 tablet.cc:2369] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:48:55.394424 23250 raft_consensus.cc:482] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:48:55.394727 23250 raft_consensus.cc:504] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 42025 } } peers { permanent_uuid: "ffc1e159ee174b918022040176b00401" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43601 } } peers { permanent_uuid: "ea2e58e0529b49859a928a494131febf" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 35121 } }
I20260808 02:48:55.396729 23250 leader_election.cc:288] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b645b415e3f5498798bbe9c98a6ba1a9 (127.25.137.195:42025), ea2e58e0529b49859a928a494131febf (127.25.137.194:35121)
I20260808 02:48:55.404218 23238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:55.397133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:59081 (local address 127.25.137.194:35121)
0808 02:48:55.397387 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:55.397394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:55.397963 (+ 569us) server_negotiation.cc:407] Connection header received
0808 02:48:55.398252 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:55.398258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:55.398346 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:55.398476 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:55.399155 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.400182 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.402343 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.403166 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.403287 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:55.403342 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:55.403851 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 02:48:55.403981 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:48:55.404268 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:55.397398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:58011 (local address 127.25.137.195:42025)
0808 02:48:55.397578 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:55.397582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:55.398733 (+ 1151us) server_negotiation.cc:407] Connection header received
0808 02:48:55.398923 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:55.398929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:55.399007 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:55.399095 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:55.399800 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.400695 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.402520 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.403363 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.403529 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:55.403599 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:55.403880 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 02:48:55.404011 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:48:55.404388 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:55.397097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:35121 (local address 127.25.137.193:59081)
0808 02:48:55.397817 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:55.397840 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:55.398013 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:55.398535 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:55.398540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:55.398570 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:55.398966 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:55.398979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.400326 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.400332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:55.402169 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:55.402183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.403423 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.403431 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:55.403632 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:55.403667 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:48:55.403750 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 02:48:55.403897 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":54,"thread_start_us":1076,"threads_started":1}
I20260808 02:48:55.404434 23255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:55.397185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:42025 (local address 127.25.137.193:58011)
0808 02:48:55.398608 (+ 1423us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:55.398630 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:55.398781 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:55.399152 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:55.399156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:55.399184 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:55.399603 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:55.399617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.400859 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.400863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:55.402337 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:55.402352 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.403569 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.403574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:55.403684 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:55.403706 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:48:55.403798 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 02:48:55.403916 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1270,"spinlock_wait_cycles":1408,"thread_start_us":315,"threads_started":1}
I20260808 02:48:55.405653 23143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f1bdb73bd584c68961f28b24cfe9b66" candidate_uuid: "ffc1e159ee174b918022040176b00401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" is_pre_election: true
I20260808 02:48:55.405647 22938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f1bdb73bd584c68961f28b24cfe9b66" candidate_uuid: "ffc1e159ee174b918022040176b00401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2e58e0529b49859a928a494131febf" is_pre_election: true
I20260808 02:48:55.406257 23143 raft_consensus.cc:2454] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffc1e159ee174b918022040176b00401 in term 0.
I20260808 02:48:55.406251 22938 raft_consensus.cc:2454] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffc1e159ee174b918022040176b00401 in term 0.
I20260808 02:48:55.406948 22615 leader_election.cc:302] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea2e58e0529b49859a928a494131febf, ffc1e159ee174b918022040176b00401; no voters:
I20260808 02:48:55.407375 23250 raft_consensus.cc:2781] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:48:55.407521 23250 raft_consensus.cc:482] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:48:55.407631 23250 raft_consensus.cc:3037] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:55.410092 23250 raft_consensus.cc:504] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 42025 } } peers { permanent_uuid: "ffc1e159ee174b918022040176b00401" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43601 } } peers { permanent_uuid: "ea2e58e0529b49859a928a494131febf" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 35121 } }
I20260808 02:48:55.410751 23250 leader_election.cc:288] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [CANDIDATE]: Term 1 election: Requested vote from peers b645b415e3f5498798bbe9c98a6ba1a9 (127.25.137.195:42025), ea2e58e0529b49859a928a494131febf (127.25.137.194:35121)
I20260808 02:48:55.411181 23143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f1bdb73bd584c68961f28b24cfe9b66" candidate_uuid: "ffc1e159ee174b918022040176b00401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b645b415e3f5498798bbe9c98a6ba1a9"
I20260808 02:48:55.411198 22938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f1bdb73bd584c68961f28b24cfe9b66" candidate_uuid: "ffc1e159ee174b918022040176b00401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2e58e0529b49859a928a494131febf"
I20260808 02:48:55.411377 23143 raft_consensus.cc:3037] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:55.411401 22938 raft_consensus.cc:3037] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:55.413618 22938 raft_consensus.cc:2454] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffc1e159ee174b918022040176b00401 in term 1.
I20260808 02:48:55.413849 23143 raft_consensus.cc:2454] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffc1e159ee174b918022040176b00401 in term 1.
I20260808 02:48:55.413986 22615 leader_election.cc:302] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea2e58e0529b49859a928a494131febf, ffc1e159ee174b918022040176b00401; no voters:
I20260808 02:48:55.414252 23250 raft_consensus.cc:2781] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:48:55.414708 23250 raft_consensus.cc:686] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 1 LEADER]: Becoming Leader. State: Replica: ffc1e159ee174b918022040176b00401, State: Running, Role: LEADER
I20260808 02:48:55.415221 23250 consensus_queue.cc:237] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 42025 } } peers { permanent_uuid: "ffc1e159ee174b918022040176b00401" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43601 } } peers { permanent_uuid: "ea2e58e0529b49859a928a494131febf" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 35121 } }
I20260808 02:48:55.419807 22492 catalog_manager.cc:5495] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffc1e159ee174b918022040176b00401 (127.25.137.193). New cstate: current_term: 1 leader_uuid: "ffc1e159ee174b918022040176b00401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 42025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffc1e159ee174b918022040176b00401" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea2e58e0529b49859a928a494131febf" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 35121 } health_report { overall_health: UNKNOWN } } }
I20260808 02:48:55.444229 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:55.438754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47884 (local address 127.25.137.193:43601)
0808 02:48:55.438952 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:55.438957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:55.438975 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:48:55.439079 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:55.439082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:55.439173 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:55.439304 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:48:55.439625 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.440577 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.442091 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.442517 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.442623 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:55.443181 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:48:55.443683 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:48:55.443829 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:55.443984 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 02:48:55.444025 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 02:48:55.452373 22938 raft_consensus.cc:1261] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf [term 1 FOLLOWER]: Refusing update from remote peer ffc1e159ee174b918022040176b00401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:48:55.452373 23143 raft_consensus.cc:1261] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9 [term 1 FOLLOWER]: Refusing update from remote peer ffc1e159ee174b918022040176b00401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:48:55.453326 23256 consensus_queue.cc:1035] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea2e58e0529b49859a928a494131febf" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 35121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:48:55.453706 23250 consensus_queue.cc:1035] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b645b415e3f5498798bbe9c98a6ba1a9" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:48:55.463490 23260 mvcc.cc:205] Tried to move back new op lower bound from 7316100446002700288 to 7316100445864304640. Current Snapshot: MvccSnapshot[applied={T|T < 7316100446002700288}]
I20260808 02:48:55.474926 22493 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:48:55.475229 22493 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:48:55.486531 22493 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3816db59d97485aac548563c59ab2c2: Sending DeleteTablet for 3 replicas of tablet 3f1bdb73bd584c68961f28b24cfe9b66
I20260808 02:48:55.488919 23115 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f1bdb73bd584c68961f28b24cfe9b66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:55 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:52256
I20260808 02:48:55.489228 22694 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f1bdb73bd584c68961f28b24cfe9b66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:55 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:47882
I20260808 02:48:55.489776 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 22584
I20260808 02:48:55.491055 22909 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f1bdb73bd584c68961f28b24cfe9b66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:55 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:51610
I20260808 02:48:55.493276 23285 tablet_replica.cc:330] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: stopping tablet replica
I20260808 02:48:55.493471 23278 tablet_replica.cc:330] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: stopping tablet replica
I20260808 02:48:55.493793 23285 raft_consensus.cc:2229] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:48:55.494338 23285 raft_consensus.cc:2258] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:55.494988 23278 raft_consensus.cc:2229] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:48:55.496522 23285 ts_tablet_manager.cc:1905] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:55.497321 23284 tablet_replica.cc:330] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: stopping tablet replica
I20260808 02:48:55.497620 23284 raft_consensus.cc:2229] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:48:55.498193 23278 raft_consensus.cc:2258] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:55.498857 23284 raft_consensus.cc:2258] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:55.500486 23278 ts_tablet_manager.cc:1905] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:55.504379 23285 ts_tablet_manager.cc:1918] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:55.504446 23284 ts_tablet_manager.cc:1905] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:55.504612 23285 log.cc:1196] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/3f1bdb73bd584c68961f28b24cfe9b66
I20260808 02:48:55.505244 23285 ts_tablet_manager.cc:1939] T 3f1bdb73bd584c68961f28b24cfe9b66 P b645b415e3f5498798bbe9c98a6ba1a9: Deleting consensus metadata
I20260808 02:48:55.507118 22476 catalog_manager.cc:4841] TS b645b415e3f5498798bbe9c98a6ba1a9 (127.25.137.195:42025): tablet 3f1bdb73bd584c68961f28b24cfe9b66 (table test-table [id=5d54a01f19f24070bc62c84924fdd47d]) successfully deleted
I20260808 02:48:55.507797 23278 ts_tablet_manager.cc:1918] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:55.507962 23278 log.cc:1196] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/3f1bdb73bd584c68961f28b24cfe9b66
I20260808 02:48:55.508517 23278 ts_tablet_manager.cc:1939] T 3f1bdb73bd584c68961f28b24cfe9b66 P ffc1e159ee174b918022040176b00401: Deleting consensus metadata
I20260808 02:48:55.510141 22476 catalog_manager.cc:4841] TS ffc1e159ee174b918022040176b00401 (127.25.137.193:43601): tablet 3f1bdb73bd584c68961f28b24cfe9b66 (table test-table [id=5d54a01f19f24070bc62c84924fdd47d]) successfully deleted
I20260808 02:48:55.511459 23284 ts_tablet_manager.cc:1918] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:55.511633 23284 log.cc:1196] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/3f1bdb73bd584c68961f28b24cfe9b66
I20260808 02:48:55.512179 23284 ts_tablet_manager.cc:1939] T 3f1bdb73bd584c68961f28b24cfe9b66 P ea2e58e0529b49859a928a494131febf: Deleting consensus metadata
Aug 08 02:48:55 dist-test-slave-k5rr krb5kdc[22429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:48:55.513720 22470 catalog_manager.cc:4841] TS ea2e58e0529b49859a928a494131febf (127.25.137.194:35121): tablet 3f1bdb73bd584c68961f28b24cfe9b66 (table test-table [id=5d54a01f19f24070bc62c84924fdd47d]) successfully deleted
I20260808 02:48:55.521787 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:55.491048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47898 (local address 127.25.137.193:43601)
0808 02:48:55.491233 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:55.491239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:55.494052 (+ 2813us) server_negotiation.cc:407] Connection header received
0808 02:48:55.494326 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:55.494333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:55.494427 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:55.494510 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:55.499301 (+ 4791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.500369 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.502331 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.503055 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.503148 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:55.514548 (+ 11400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:55.514564 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:55.514583 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:55.514634 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:55.518508 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:55.519354 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:55.519364 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:55.519367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:55.519552 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:55.520028 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:55.520034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:55.520038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:55.520499 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:55.520812 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:55.521159 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 02:48:55.521414 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:48:55.522639 22793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:55.622207 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 22584
I20260808 02:48:55.649197 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 22809
Aug 08 02:48:55 dist-test-slave-k5rr krb5kdc[22429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:48:55.663935 23238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:55.649957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51620 (local address 127.25.137.194:35121)
0808 02:48:55.650150 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:55.650156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:55.650177 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:48:55.650372 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:55.650377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:55.650449 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:55.650534 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:55.651877 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.652580 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.653976 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.654371 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.654482 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:55.658561 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:55.658576 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:55.658592 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:55.658625 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:55.661239 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:55.661885 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:55.661892 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:55.661895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:55.662029 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:55.662406 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:55.662410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:55.662412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:55.662785 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:55.663082 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:55.663371 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 02:48:55.663603 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:48:55.664573 22997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:55.752892 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 22809
I20260808 02:48:55.777179 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 23015
Aug 08 02:48:55 dist-test-slave-k5rr krb5kdc[22429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:48:55.791623 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:55.778007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52268 (local address 127.25.137.195:42025)
0808 02:48:55.778209 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:55.778214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:55.778235 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:48:55.778372 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:55.778376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:55.778453 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:55.778535 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:55.779826 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.780545 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.781867 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:55.782254 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:55.782344 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:55.786473 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:55.786488 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:55.786505 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:48:55.786537 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:55.789244 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:55.789891 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:55.789899 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:55.789901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:55.790035 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:48:55.790336 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:48:55.790339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:48:55.790341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:48:55.790680 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:55.790899 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:55.791089 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 02:48:55.791254 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:48:55.792315 23209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:55.880348 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 23015
I20260808 02:48:55.903893 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 22448
I20260808 02:48:55.904979 22539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:56.015692 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 22448
2026-08-08T02:48:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4794 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T02:48:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:48:56Z Disabled control of system clock
I20260808 02:48:56.111915 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:39743
--webserver_interface=127.25.137.254
--webserver_port=0
--builtin_ntp_servers=127.25.137.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:39743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:48:56.451519 23302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:56.451810 23302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:56.451893 23302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:56.458933 23302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:48:56.459038 23302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:56.459076 23302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:56.459133 23302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:48:56.459177 23302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:48:56.468235 23302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:39743
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:39743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:56.469486 23302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:56.471359 23302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:56.482492 23316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:56.482571 23309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:56.482542 23308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:56.483207 23302 server_base.cc:1034] running on GCE node
I20260808 02:48:56.483966 23302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:56.485363 23302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:56.486574 23302 hybrid_clock.cc:648] HybridClock initialized: now 1786157336486518 us; error 46 us; skew 500 ppm
I20260808 02:48:56.486915 23302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:48:56.491767 23302 webserver.cc:460] Webserver started at http://127.25.137.254:34723/ using document root <none> and password file <none>
I20260808 02:48:56.492477 23302 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:56.492580 23302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:56.492908 23302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:56.495806 23302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "f8da57c515634490a6cbde2e12d73b56"
format_stamp: "Formatted at 2026-08-08 02:48:56 on dist-test-slave-k5rr"
server_key: "23cd94d994cffc3fe2ccbc80c40f387b"
server_key_iv: "1ad8114bb48403af73c5f359de687fec"
server_key_version: "encryptionkey@0"
I20260808 02:48:56.496621 23302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "f8da57c515634490a6cbde2e12d73b56"
format_stamp: "Formatted at 2026-08-08 02:48:56 on dist-test-slave-k5rr"
server_key: "23cd94d994cffc3fe2ccbc80c40f387b"
server_key_iv: "1ad8114bb48403af73c5f359de687fec"
server_key_version: "encryptionkey@0"
I20260808 02:48:56.501637 23302 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 02:48:56.505162 23328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:56.506516 23302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:48:56.506696 23302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "f8da57c515634490a6cbde2e12d73b56"
format_stamp: "Formatted at 2026-08-08 02:48:56 on dist-test-slave-k5rr"
server_key: "23cd94d994cffc3fe2ccbc80c40f387b"
server_key_iv: "1ad8114bb48403af73c5f359de687fec"
server_key_version: "encryptionkey@0"
I20260808 02:48:56.506910 23302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:56.521936 23302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:56.522722 23302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:56.522990 23302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:56.540664 23302 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:39743
I20260808 02:48:56.540683 23400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:39743 every 8 connection(s)
I20260808 02:48:56.542068 23302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:48:56.545714 23406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:56.551874 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 23302
I20260808 02:48:56.552050 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:48:56.552619 26151 external_mini_cluster.cc:1428] Setting key 09e7bef3bee5d615c8e696aaee251251
I20260808 02:48:56.556686 23406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56: Bootstrap starting.
I20260808 02:48:56.560003 23406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:56.561293 23406 log.cc:824] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:56.564579 23406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56: No bootstrap required, opened a new log
I20260808 02:48:56.566756 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:56.554588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.25.137.254:39743)
0808 02:48:56.555572 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:56.555583 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:56.555624 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:48:56.557393 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:56.557416 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:56.557840 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:56.558159 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:56.558804 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:56.560089 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:56.562100 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:56.562996 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:56.563131 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:56.564422 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:56.564433 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:56.564452 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:56.565036 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:56.565125 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:56.565157 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:56.565249 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:56.565490 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 02:48:56.565790 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":355,"threads_started":1}
I20260808 02:48:56.572204 23406 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8da57c515634490a6cbde2e12d73b56" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39743 } }
I20260808 02:48:56.572561 23406 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:56.572667 23406 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8da57c515634490a6cbde2e12d73b56, State: Initialized, Role: FOLLOWER
I20260808 02:48:56.573237 23406 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8da57c515634490a6cbde2e12d73b56" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39743 } }
I20260808 02:48:56.573468 23406 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:48:56.573596 23406 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:48:56.573750 23406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:56.575879 23406 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8da57c515634490a6cbde2e12d73b56" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39743 } }
I20260808 02:48:56.576342 23406 leader_election.cc:302] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8da57c515634490a6cbde2e12d73b56; no voters:
I20260808 02:48:56.576880 23406 leader_election.cc:288] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:48:56.577026 23413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:48:56.577711 23413 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [term 1 LEADER]: Becoming Leader. State: Replica: f8da57c515634490a6cbde2e12d73b56, State: Running, Role: LEADER
I20260808 02:48:56.578204 23413 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8da57c515634490a6cbde2e12d73b56" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39743 } }
I20260808 02:48:56.578966 23406 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:48:56.581022 23414 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8da57c515634490a6cbde2e12d73b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8da57c515634490a6cbde2e12d73b56" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39743 } } }
I20260808 02:48:56.581353 23416 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8da57c515634490a6cbde2e12d73b56. Latest consensus state: current_term: 1 leader_uuid: "f8da57c515634490a6cbde2e12d73b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8da57c515634490a6cbde2e12d73b56" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39743 } } }
I20260808 02:48:56.581524 23414 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:56.581902 23416 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56 [sys.catalog]: This master's current role is: LEADER
I20260808 02:48:56.587756 23427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:48:56.593050 23427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:48:56.601352 23427 catalog_manager.cc:1347] Generated new cluster ID: 922bafb9acfb46618b10a9462598efb6
I20260808 02:48:56.601473 23427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:48:56.641340 23427 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:48:56.642607 23427 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:48:56.657874 23427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56: Generated new TSK 0
I20260808 02:48:56.658730 23427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:48:56.669327 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39743
--builtin_ntp_servers=127.25.137.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:48:57.017645 23443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:57.017927 23443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:57.018057 23443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:57.025090 23443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:57.025180 23443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:57.025318 23443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:48:57.034257 23443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:57.035517 23443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:57.037288 23443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:57.050290 23455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:57.050480 23450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:57.050457 23452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:57.051342 23443 server_base.cc:1034] running on GCE node
I20260808 02:48:57.051949 23443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:57.052824 23443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:57.054030 23443 hybrid_clock.cc:648] HybridClock initialized: now 1786157337053971 us; error 49 us; skew 500 ppm
I20260808 02:48:57.054323 23443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:48:57.056624 23443 webserver.cc:460] Webserver started at http://127.25.137.193:40927/ using document root <none> and password file <none>
I20260808 02:48:57.057327 23443 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:57.057418 23443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:57.057680 23443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:57.060544 23443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "82d9c7c069f54705ad396117d518c67b"
format_stamp: "Formatted at 2026-08-08 02:48:57 on dist-test-slave-k5rr"
server_key: "c0c5bc087b1048cd99625ea75516080f"
server_key_iv: "1df6abc774fc784bcff917f40fc10b12"
server_key_version: "encryptionkey@0"
I20260808 02:48:57.061357 23443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "82d9c7c069f54705ad396117d518c67b"
format_stamp: "Formatted at 2026-08-08 02:48:57 on dist-test-slave-k5rr"
server_key: "c0c5bc087b1048cd99625ea75516080f"
server_key_iv: "1df6abc774fc784bcff917f40fc10b12"
server_key_version: "encryptionkey@0"
I20260808 02:48:57.066562 23443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 02:48:57.069990 23463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:57.071352 23443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:48:57.071521 23443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "82d9c7c069f54705ad396117d518c67b"
format_stamp: "Formatted at 2026-08-08 02:48:57 on dist-test-slave-k5rr"
server_key: "c0c5bc087b1048cd99625ea75516080f"
server_key_iv: "1df6abc774fc784bcff917f40fc10b12"
server_key_version: "encryptionkey@0"
I20260808 02:48:57.071724 23443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:57.084709 23443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:57.085515 23443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:57.085791 23443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:57.086833 23443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:57.088376 23443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:57.088462 23443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:57.088565 23443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:57.088637 23443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:57.118851 23443 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:42171
I20260808 02:48:57.118902 23632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:42171 every 8 connection(s)
I20260808 02:48:57.120131 23443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:48:57.126925 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 23443
I20260808 02:48:57.127100 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:48:57.127655 26151 external_mini_cluster.cc:1428] Setting key eaef9622513a62e7b348748d7f3c2225
I20260808 02:48:57.130695 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39743
--builtin_ntp_servers=127.25.137.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:48:57.133378 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:57.122637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:35983 (local address 127.25.137.254:39743)
0808 02:48:57.123119 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:57.123125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:57.124276 (+ 1151us) server_negotiation.cc:407] Connection header received
0808 02:48:57.126055 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:57.126060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:57.126129 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:57.126226 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:57.127531 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:57.128355 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:57.130339 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:57.130969 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:57.131089 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:57.132423 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:57.132429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:57.132434 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:57.132457 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:57.132508 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:57.132523 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:57.132607 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:57.132943 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 02:48:57.133060 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":219,"threads_started":1}
I20260808 02:48:57.134421 23636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:57.123239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:39743 (local address 127.25.137.193:35983)
0808 02:48:57.124116 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:57.124152 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:57.125787 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:57.126413 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:57.126420 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:57.126517 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:57.127351 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:57.127379 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:57.128499 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:57.128503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:57.130178 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:57.130192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:57.131189 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:57.131195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:57.131401 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:57.132091 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:48:57.132094 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:57.132141 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:48:57.132233 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:57.132662 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:57.132673 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:57.132681 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:48:57.132918 (+ 237us) client_negotiation.cc:240] Negotiation successful
0808 02:48:57.133276 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":260,"threads_started":1}
I20260808 02:48:57.136014 23633 heartbeater.cc:347] Connected to a master server at 127.25.137.254:39743
I20260808 02:48:57.136325 23633 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:57.137094 23633 heartbeater.cc:511] Master 127.25.137.254:39743 requested a full tablet report, sending...
I20260808 02:48:57.139117 23356 ts_manager.cc:194] Registered new tserver with Master: 82d9c7c069f54705ad396117d518c67b (127.25.137.193:42171)
I20260808 02:48:57.140592 23356 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.193:35983
W20260808 02:48:57.473942 23637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:57.474239 23637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:57.474334 23637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:57.483834 23637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:57.483970 23637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:57.484210 23637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:48:57.494110 23637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:57.495483 23637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:57.497469 23637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:57.511277 23644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:57.511219 23646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:57.511238 23648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:57.511945 23637 server_base.cc:1034] running on GCE node
I20260808 02:48:57.512634 23637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:57.513614 23637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:57.514850 23637 hybrid_clock.cc:648] HybridClock initialized: now 1786157337514756 us; error 49 us; skew 500 ppm
I20260808 02:48:57.515156 23637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:48:57.517469 23637 webserver.cc:460] Webserver started at http://127.25.137.194:43773/ using document root <none> and password file <none>
I20260808 02:48:57.518144 23637 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:57.518239 23637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:57.518541 23637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:57.521374 23637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "e019bc58cf7e4da8a410e6d71b2e347c"
format_stamp: "Formatted at 2026-08-08 02:48:57 on dist-test-slave-k5rr"
server_key: "64e703d3f0da1c406f51d73e6aa65533"
server_key_iv: "263d21b4181d561e79ec4b2b1f98c10e"
server_key_version: "encryptionkey@0"
I20260808 02:48:57.522131 23637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "e019bc58cf7e4da8a410e6d71b2e347c"
format_stamp: "Formatted at 2026-08-08 02:48:57 on dist-test-slave-k5rr"
server_key: "64e703d3f0da1c406f51d73e6aa65533"
server_key_iv: "263d21b4181d561e79ec4b2b1f98c10e"
server_key_version: "encryptionkey@0"
I20260808 02:48:57.527043 23637 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:48:57.530654 23659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:57.532017 23637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:48:57.532171 23637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "e019bc58cf7e4da8a410e6d71b2e347c"
format_stamp: "Formatted at 2026-08-08 02:48:57 on dist-test-slave-k5rr"
server_key: "64e703d3f0da1c406f51d73e6aa65533"
server_key_iv: "263d21b4181d561e79ec4b2b1f98c10e"
server_key_version: "encryptionkey@0"
I20260808 02:48:57.532359 23637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:57.544245 23637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:57.545231 23637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:57.545578 23637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:57.546715 23637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:57.548419 23637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:57.548521 23637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:57.548620 23637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:57.548687 23637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:57.580461 23637 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:36859
I20260808 02:48:57.580518 23825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:36859 every 8 connection(s)
I20260808 02:48:57.581928 23637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:48:57.583541 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 23637
I20260808 02:48:57.583722 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:48:57.584244 26151 external_mini_cluster.cc:1428] Setting key 4ecd29f9daf0366a457bfd14408c7f19
I20260808 02:48:57.587184 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39743
--builtin_ntp_servers=127.25.137.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:48:57.594825 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:57.584598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:41151 (local address 127.25.137.254:39743)
0808 02:48:57.584803 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:57.584808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:57.586458 (+ 1650us) server_negotiation.cc:407] Connection header received
0808 02:48:57.588299 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:57.588306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:57.588387 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:57.588511 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:57.589593 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:57.590566 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:57.592083 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:57.592604 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:57.592715 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:57.594061 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:57.594067 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:57.594071 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:57.594092 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:57.594130 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:57.594146 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:57.594240 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:57.594446 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 02:48:57.594540 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:48:57.595885 23834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:57.585268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:39743 (local address 127.25.137.194:41151)
0808 02:48:57.586300 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:57.586344 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:57.588043 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:57.588694 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:57.588703 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:57.588796 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:57.589437 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:57.589454 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:57.590697 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:57.590700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:57.591944 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:57.591957 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:57.592804 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:57.592808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:57.593083 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:57.593715 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:48:57.593717 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:57.593760 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:48:57.593864 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:57.594253 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:57.594261 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:57.594267 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:48:57.594410 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 02:48:57.594796 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":305,"threads_started":1}
I20260808 02:48:57.597452 23831 heartbeater.cc:347] Connected to a master server at 127.25.137.254:39743
I20260808 02:48:57.597728 23831 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:57.598433 23831 heartbeater.cc:511] Master 127.25.137.254:39743 requested a full tablet report, sending...
I20260808 02:48:57.599798 23356 ts_manager.cc:194] Registered new tserver with Master: e019bc58cf7e4da8a410e6d71b2e347c (127.25.137.194:36859)
I20260808 02:48:57.600605 23356 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.194:41151
W20260808 02:48:57.941547 23835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:48:57.941876 23835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:48:57.941972 23835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:48:57.949421 23835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:48:57.949545 23835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:48:57.949674 23835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:48:57.958989 23835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:39743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:48:57.960345 23835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:48:57.962206 23835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:48:57.975590 23846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:57.975551 23849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:48:57.975620 23845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:48:57.976440 23835 server_base.cc:1034] running on GCE node
I20260808 02:48:57.977114 23835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:48:57.978091 23835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:48:57.979305 23835 hybrid_clock.cc:648] HybridClock initialized: now 1786157337979243 us; error 52 us; skew 500 ppm
I20260808 02:48:57.979624 23835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:48:57.981981 23835 webserver.cc:460] Webserver started at http://127.25.137.195:41249/ using document root <none> and password file <none>
I20260808 02:48:57.982632 23835 fs_manager.cc:359] Metadata directory not provided
I20260808 02:48:57.982720 23835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:48:57.983026 23835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:48:57.986112 23835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "2b3568c18e164f26ac3ee6fdcc4f9849"
format_stamp: "Formatted at 2026-08-08 02:48:57 on dist-test-slave-k5rr"
server_key: "acf33a778f41434a1f51567da3913b64"
server_key_iv: "90266ec1c459e67e3431d4155c0add26"
server_key_version: "encryptionkey@0"
I20260808 02:48:57.986896 23835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "2b3568c18e164f26ac3ee6fdcc4f9849"
format_stamp: "Formatted at 2026-08-08 02:48:57 on dist-test-slave-k5rr"
server_key: "acf33a778f41434a1f51567da3913b64"
server_key_iv: "90266ec1c459e67e3431d4155c0add26"
server_key_version: "encryptionkey@0"
I20260808 02:48:57.992130 23835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:48:57.995582 23858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:57.996897 23835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:48:57.997066 23835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "2b3568c18e164f26ac3ee6fdcc4f9849"
format_stamp: "Formatted at 2026-08-08 02:48:57 on dist-test-slave-k5rr"
server_key: "acf33a778f41434a1f51567da3913b64"
server_key_iv: "90266ec1c459e67e3431d4155c0add26"
server_key_version: "encryptionkey@0"
I20260808 02:48:57.997254 23835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:48:58.015614 23835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:48:58.016423 23835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:48:58.016693 23835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:48:58.017725 23835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:48:58.019187 23835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:48:58.019277 23835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:58.019358 23835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:48:58.019418 23835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:48:58.050606 23835 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:36213
I20260808 02:48:58.050666 24021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:36213 every 8 connection(s)
I20260808 02:48:58.051935 23835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:48:58.061403 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 23835
I20260808 02:48:58.061550 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:48:58.062036 26151 external_mini_cluster.cc:1428] Setting key 86d9105da56b6960357b7c5789bb114e
I20260808 02:48:58.064376 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.054460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:32885 (local address 127.25.137.254:39743)
0808 02:48:58.054646 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:58.054651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:58.055996 (+ 1345us) server_negotiation.cc:407] Connection header received
0808 02:48:58.057829 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:58.057834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:58.057913 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:58.058005 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:58.059027 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.059775 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.061366 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.061817 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.061928 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.063490 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:58.063496 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:58.063500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:58.063522 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:58.063563 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:58.063580 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:58.063674 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:58.063942 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 02:48:58.064052 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:48:58.065479 24025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.054949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:39743 (local address 127.25.137.195:32885)
0808 02:48:58.055844 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.055880 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:58.057553 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:58.058237 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:58.058245 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:58.058328 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:48:58.058870 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.058886 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.059917 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.059920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.061227 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.061241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.062043 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.062050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.062304 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.063124 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:48:58.063128 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:48:58.063178 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:48:58.063286 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:48:58.063674 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:48:58.063682 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:48:58.063688 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:48:58.063929 (+ 241us) client_negotiation.cc:240] Negotiation successful
0808 02:48:58.064294 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":265,"threads_started":1}
I20260808 02:48:58.066715 24022 heartbeater.cc:347] Connected to a master server at 127.25.137.254:39743
I20260808 02:48:58.067034 24022 heartbeater.cc:464] Registering TS with master...
I20260808 02:48:58.067754 24022 heartbeater.cc:511] Master 127.25.137.254:39743 requested a full tablet report, sending...
I20260808 02:48:58.069224 23356 ts_manager.cc:194] Registered new tserver with Master: 2b3568c18e164f26ac3ee6fdcc4f9849 (127.25.137.195:36213)
I20260808 02:48:58.070209 23356 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.195:32885
I20260808 02:48:58.077631 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:48:58.087986 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.081882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52580 (local address 127.25.137.254:39743)
0808 02:48:58.082038 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:58.082042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:58.082510 (+ 468us) server_negotiation.cc:407] Connection header received
0808 02:48:58.082644 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:58.082648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:58.082713 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:58.082792 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:58.083268 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.084050 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.085577 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.085997 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.086106 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.087279 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:58.087284 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:58.087287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:58.087313 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:58.087347 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:58.087359 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:58.087471 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:58.087683 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 02:48:58.087737 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:48:58.091902 23356 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:48:58.098562 23356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:48:58.133782 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.111063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34850 (local address 127.25.137.193:42171)
0808 02:48:58.124004 (+ 12941us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:58.124014 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:58.124827 (+ 813us) server_negotiation.cc:407] Connection header received
0808 02:48:58.125152 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:58.125159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:58.125448 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:58.125612 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:58.126731 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.128285 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.131176 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.132354 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.132561 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.132802 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:58.133146 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 02:48:58.133332 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12761,"thread_start_us":15304,"threads_started":1}
I20260808 02:48:58.134302 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.110840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40946 (local address 127.25.137.194:36859)
0808 02:48:58.125066 (+ 14226us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:58.125078 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:58.125114 (+ 36us) server_negotiation.cc:407] Connection header received
0808 02:48:58.125232 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:58.125253 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:58.125509 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:58.125723 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:58.126883 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.128454 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.131381 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.132762 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.132915 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.133131 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:58.133431 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 02:48:58.133600 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14011,"thread_start_us":17154,"threads_started":1}
I20260808 02:48:58.134982 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.110974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42171 (local address 127.0.0.1:34850)
0808 02:48:58.124710 (+ 13736us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.124733 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:58.124935 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:58.125915 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:58.125922 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:58.125949 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:58.126501 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.126521 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.128410 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.128416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.130959 (+ 2543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.130975 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.132563 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.132570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.132810 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.132869 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 02:48:58.134022 (+ 1153us) client_negotiation.cc:240] Negotiation successful
0808 02:48:58.134543 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13551,"mutex_wait_us":368,"thread_start_us":11973,"threads_started":1}
I20260808 02:48:58.135327 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.110790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:36859 (local address 127.0.0.1:40946)
0808 02:48:58.123441 (+ 12651us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.123498 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:58.123693 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:58.126048 (+ 2355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:58.126054 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:58.126093 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:58.126645 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.126661 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.128602 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.128607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.131205 (+ 2598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.131216 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.132976 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.132984 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.133214 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.133271 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 02:48:58.133363 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 02:48:58.134157 (+ 794us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12347,"mutex_wait_us":459,"spinlock_wait_cycles":1920,"thread_start_us":12336,"threads_started":1}
I20260808 02:48:58.136288 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.124095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:36213 (local address 127.0.0.1:51346)
0808 02:48:58.128309 (+ 4214us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.128333 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:58.128475 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:58.129445 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:58.129453 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:58.129486 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:58.130000 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.130015 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.131662 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.131667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.133591 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.133606 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.135341 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.135348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.135777 (+ 429us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.135810 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:48:58.135903 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 02:48:58.136022 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4083,"thread_start_us":297,"threads_started":1}
I20260808 02:48:58.136533 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.124198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.25.137.195:36213)
0808 02:48:58.128107 (+ 3909us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:58.128124 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:58.128621 (+ 497us) server_negotiation.cc:407] Connection header received
0808 02:48:58.128708 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:58.128713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:58.128916 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:58.129131 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:58.130270 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.131506 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.133809 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.135080 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.135276 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.135486 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:58.135967 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 02:48:58.136153 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3748,"thread_start_us":3600,"threads_started":1}
I20260808 02:48:58.138872 23537 tablet_service.cc:1467] Processing CreateTablet for tablet a36162a331ed4ddca20cab8c590a4f4f (DEFAULT_TABLE table=test-table [id=71edc29663af452e98a81be8d8979c05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:58.139103 23731 tablet_service.cc:1467] Processing CreateTablet for tablet a36162a331ed4ddca20cab8c590a4f4f (DEFAULT_TABLE table=test-table [id=71edc29663af452e98a81be8d8979c05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:58.140430 23537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a36162a331ed4ddca20cab8c590a4f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:58.140451 23731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a36162a331ed4ddca20cab8c590a4f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:58.140055 23927 tablet_service.cc:1467] Processing CreateTablet for tablet a36162a331ed4ddca20cab8c590a4f4f (DEFAULT_TABLE table=test-table [id=71edc29663af452e98a81be8d8979c05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:48:58.141462 23927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a36162a331ed4ddca20cab8c590a4f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:48:58.143411 23633 heartbeater.cc:503] Master 127.25.137.254:39743 was elected leader, sending a full tablet report...
I20260808 02:48:58.152118 24051 tablet_bootstrap.cc:492] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b: Bootstrap starting.
I20260808 02:48:58.154480 24053 tablet_bootstrap.cc:492] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c: Bootstrap starting.
I20260808 02:48:58.155936 24055 tablet_bootstrap.cc:492] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: Bootstrap starting.
I20260808 02:48:58.157239 24051 tablet_bootstrap.cc:654] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:58.157883 24053 tablet_bootstrap.cc:654] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:58.158511 24051 log.cc:824] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:58.158672 24055 tablet_bootstrap.cc:654] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: Neither blocks nor log segments found. Creating new log.
I20260808 02:48:58.159075 24053 log.cc:824] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:58.159751 24055 log.cc:824] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: Log is configured to *not* fsync() on all Append() calls
I20260808 02:48:58.161877 24051 tablet_bootstrap.cc:492] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b: No bootstrap required, opened a new log
I20260808 02:48:58.162166 24051 ts_tablet_manager.cc:1397] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260808 02:48:58.162392 24055 tablet_bootstrap.cc:492] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: No bootstrap required, opened a new log
I20260808 02:48:58.162482 24053 tablet_bootstrap.cc:492] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c: No bootstrap required, opened a new log
I20260808 02:48:58.162657 24055 ts_tablet_manager.cc:1397] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:48:58.162744 24053 ts_tablet_manager.cc:1397] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.006s
I20260808 02:48:58.169831 24055 raft_consensus.cc:348] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36859 } } peers { permanent_uuid: "82d9c7c069f54705ad396117d518c67b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42171 } } peers { permanent_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36213 } }
I20260808 02:48:58.170270 24055 raft_consensus.cc:374] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:58.170370 24055 raft_consensus.cc:729] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b3568c18e164f26ac3ee6fdcc4f9849, State: Initialized, Role: FOLLOWER
I20260808 02:48:58.170969 24055 consensus_queue.cc:260] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36859 } } peers { permanent_uuid: "82d9c7c069f54705ad396117d518c67b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42171 } } peers { permanent_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36213 } }
I20260808 02:48:58.172103 24022 heartbeater.cc:503] Master 127.25.137.254:39743 was elected leader, sending a full tablet report...
I20260808 02:48:58.172824 24055 ts_tablet_manager.cc:1428] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260808 02:48:58.172838 24051 raft_consensus.cc:348] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36859 } } peers { permanent_uuid: "82d9c7c069f54705ad396117d518c67b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42171 } } peers { permanent_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36213 } }
I20260808 02:48:58.172961 24053 raft_consensus.cc:348] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36859 } } peers { permanent_uuid: "82d9c7c069f54705ad396117d518c67b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42171 } } peers { permanent_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36213 } }
I20260808 02:48:58.173358 24051 raft_consensus.cc:374] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:58.173380 24053 raft_consensus.cc:374] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:48:58.173465 24053 raft_consensus.cc:729] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e019bc58cf7e4da8a410e6d71b2e347c, State: Initialized, Role: FOLLOWER
I20260808 02:48:58.173465 24051 raft_consensus.cc:729] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82d9c7c069f54705ad396117d518c67b, State: Initialized, Role: FOLLOWER
I20260808 02:48:58.174271 24051 consensus_queue.cc:260] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36859 } } peers { permanent_uuid: "82d9c7c069f54705ad396117d518c67b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42171 } } peers { permanent_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36213 } }
I20260808 02:48:58.174522 24053 consensus_queue.cc:260] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36859 } } peers { permanent_uuid: "82d9c7c069f54705ad396117d518c67b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42171 } } peers { permanent_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36213 } }
I20260808 02:48:58.177157 23831 heartbeater.cc:503] Master 127.25.137.254:39743 was elected leader, sending a full tablet report...
I20260808 02:48:58.177946 24053 ts_tablet_manager.cc:1428] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260808 02:48:58.179513 24051 ts_tablet_manager.cc:1428] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b: Time spent starting tablet: real 0.017s user 0.010s sys 0.009s
I20260808 02:48:58.219429 24068 raft_consensus.cc:482] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:48:58.219734 24068 raft_consensus.cc:504] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36859 } } peers { permanent_uuid: "82d9c7c069f54705ad396117d518c67b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42171 } } peers { permanent_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36213 } }
I20260808 02:48:58.221545 24068 leader_election.cc:288] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82d9c7c069f54705ad396117d518c67b (127.25.137.193:42171), 2b3568c18e164f26ac3ee6fdcc4f9849 (127.25.137.195:36213)
I20260808 02:48:58.230602 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.222054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:45533 (local address 127.25.137.195:36213)
0808 02:48:58.222212 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:58.222219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:58.223017 (+ 798us) server_negotiation.cc:407] Connection header received
0808 02:48:58.223355 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:58.223361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:58.223440 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:58.223532 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:58.224190 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.225365 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.228312 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.229598 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.229778 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.229848 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:58.230214 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 02:48:58.230363 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:48:58.230602 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.222013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:40971 (local address 127.25.137.193:42171)
0808 02:48:58.222213 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:58.222219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:58.223352 (+ 1133us) server_negotiation.cc:407] Connection header received
0808 02:48:58.223484 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:58.223488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:58.223582 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:58.223662 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:48:58.224404 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.225443 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.228308 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.229598 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.229771 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.229888 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:58.230215 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 02:48:58.230350 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:48:58.230816 24069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.222062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42171 (local address 127.25.137.194:40971)
0808 02:48:58.223237 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.223252 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:58.223372 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:58.223737 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:58.223745 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:58.223797 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:58.224241 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.224251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.225571 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.225579 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.228057 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.228073 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.229839 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.229846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.230035 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.230082 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 02:48:58.230174 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 02:48:58.230352 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"mutex_wait_us":218,"spinlock_wait_cycles":46208,"thread_start_us":414,"threads_started":1}
I20260808 02:48:58.230827 24070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.222038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:36213 (local address 127.25.137.194:45533)
0808 02:48:58.222892 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.222916 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:48:58.223151 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:48:58.223573 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:48:58.223578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:48:58.223603 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:48:58.224031 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.224044 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.225511 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.225517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.228058 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:48:58.228073 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.229804 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.229812 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:48:58.230053 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.230098 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 02:48:58.230174 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:48:58.230350 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":232,"thread_start_us":883,"threads_started":1}
I20260808 02:48:58.231871 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36162a331ed4ddca20cab8c590a4f4f" candidate_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" is_pre_election: true
I20260808 02:48:58.231874 23571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36162a331ed4ddca20cab8c590a4f4f" candidate_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82d9c7c069f54705ad396117d518c67b" is_pre_election: true
I20260808 02:48:58.232355 23571 raft_consensus.cc:2454] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e019bc58cf7e4da8a410e6d71b2e347c in term 0.
I20260808 02:48:58.232350 23960 raft_consensus.cc:2454] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e019bc58cf7e4da8a410e6d71b2e347c in term 0.
I20260808 02:48:58.232954 23666 leader_election.cc:302] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82d9c7c069f54705ad396117d518c67b, e019bc58cf7e4da8a410e6d71b2e347c; no voters:
I20260808 02:48:58.233340 24068 raft_consensus.cc:2781] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:48:58.233474 24068 raft_consensus.cc:482] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:48:58.233568 24068 raft_consensus.cc:3037] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:58.235877 24068 raft_consensus.cc:504] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36859 } } peers { permanent_uuid: "82d9c7c069f54705ad396117d518c67b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42171 } } peers { permanent_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36213 } }
I20260808 02:48:58.236485 24068 leader_election.cc:288] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [CANDIDATE]: Term 1 election: Requested vote from peers 82d9c7c069f54705ad396117d518c67b (127.25.137.193:42171), 2b3568c18e164f26ac3ee6fdcc4f9849 (127.25.137.195:36213)
I20260808 02:48:58.236862 23571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36162a331ed4ddca20cab8c590a4f4f" candidate_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82d9c7c069f54705ad396117d518c67b"
I20260808 02:48:58.236888 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36162a331ed4ddca20cab8c590a4f4f" candidate_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849"
I20260808 02:48:58.237054 23571 raft_consensus.cc:3037] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:58.237084 23960 raft_consensus.cc:3037] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:48:58.239240 23960 raft_consensus.cc:2454] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e019bc58cf7e4da8a410e6d71b2e347c in term 1.
I20260808 02:48:58.239307 23571 raft_consensus.cc:2454] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e019bc58cf7e4da8a410e6d71b2e347c in term 1.
I20260808 02:48:58.239611 23662 leader_election.cc:302] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b3568c18e164f26ac3ee6fdcc4f9849, e019bc58cf7e4da8a410e6d71b2e347c; no voters:
I20260808 02:48:58.239919 24068 raft_consensus.cc:2781] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:48:58.240330 24068 raft_consensus.cc:686] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 1 LEADER]: Becoming Leader. State: Replica: e019bc58cf7e4da8a410e6d71b2e347c, State: Running, Role: LEADER
I20260808 02:48:58.240950 24068 consensus_queue.cc:237] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36859 } } peers { permanent_uuid: "82d9c7c069f54705ad396117d518c67b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42171 } } peers { permanent_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36213 } }
I20260808 02:48:58.245396 23353 catalog_manager.cc:5495] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c reported cstate change: term changed from 0 to 1, leader changed from <none> to e019bc58cf7e4da8a410e6d71b2e347c (127.25.137.194). New cstate: current_term: 1 leader_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019bc58cf7e4da8a410e6d71b2e347c" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 36859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82d9c7c069f54705ad396117d518c67b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20260808 02:48:58.296854 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.289891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40954 (local address 127.25.137.194:36859)
0808 02:48:58.290252 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:58.290259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:58.290282 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:48:58.290498 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:58.290504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:58.290615 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:58.290778 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:48:58.291222 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.292073 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.294266 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.294727 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.294831 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.295464 (+ 633us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:48:58.296155 (+ 691us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:48:58.296352 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:58.296501 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 02:48:58.296549 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
W20260808 02:48:58.304442 24023 tablet.cc:2369] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:48:58.309320 23960 raft_consensus.cc:1261] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849 [term 1 FOLLOWER]: Refusing update from remote peer e019bc58cf7e4da8a410e6d71b2e347c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:48:58.309377 23571 raft_consensus.cc:1261] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b [term 1 FOLLOWER]: Refusing update from remote peer e019bc58cf7e4da8a410e6d71b2e347c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:48:58.310379 24072 consensus_queue.cc:1035] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [LEADER]: Connected to new peer: Peer: permanent_uuid: "82d9c7c069f54705ad396117d518c67b" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 42171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:48:58.310818 24068 consensus_queue.cc:1035] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b3568c18e164f26ac3ee6fdcc4f9849" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:48:58.329885 24077 mvcc.cc:205] Tried to move back new op lower bound from 7316100457703260160 to 7316100457438937088. Current Snapshot: MvccSnapshot[applied={T|T < 7316100457703260160}]
I20260808 02:48:58.330199 24080 mvcc.cc:205] Tried to move back new op lower bound from 7316100457703260160 to 7316100457438937088. Current Snapshot: MvccSnapshot[applied={T|T < 7316100457703260160}]
I20260808 02:48:58.343832 23353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:48:58.344110 23353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:48:58.348062 23353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8da57c515634490a6cbde2e12d73b56: Sending DeleteTablet for 3 replicas of tablet a36162a331ed4ddca20cab8c590a4f4f
I20260808 02:48:58.349391 23537 tablet_service.cc:1514] Processing DeleteTablet for tablet a36162a331ed4ddca20cab8c590a4f4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:58 UTC) from {username='slave'} at 127.0.0.1:34850
I20260808 02:48:58.349395 23731 tablet_service.cc:1514] Processing DeleteTablet for tablet a36162a331ed4ddca20cab8c590a4f4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:58 UTC) from {username='slave'} at 127.0.0.1:40946
I20260808 02:48:58.349377 23927 tablet_service.cc:1514] Processing DeleteTablet for tablet a36162a331ed4ddca20cab8c590a4f4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:48:58 UTC) from {username='slave'} at 127.0.0.1:51346
I20260808 02:48:58.351735 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 23443
I20260808 02:48:58.361959 24091 tablet_replica.cc:330] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: stopping tablet replica
I20260808 02:48:58.362440 24091 raft_consensus.cc:2229] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:48:58.363011 24091 raft_consensus.cc:2258] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:58.363863 24094 tablet_replica.cc:330] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b: stopping tablet replica
I20260808 02:48:58.364274 24094 raft_consensus.cc:2229] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:48:58.364770 24094 raft_consensus.cc:2258] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:58.367040 24094 ts_tablet_manager.cc:1905] T a36162a331ed4ddca20cab8c590a4f4f P 82d9c7c069f54705ad396117d518c67b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:58.372223 24091 ts_tablet_manager.cc:1905] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:58.373251 24095 tablet_replica.cc:330] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c: stopping tablet replica
I20260808 02:48:58.373647 24095 raft_consensus.cc:2229] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:48:58.374419 24095 raft_consensus.cc:2258] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:48:58.375417 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.353857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.25.137.193:42171)
0808 02:48:58.354073 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:58.354081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:58.365537 (+ 11456us) server_negotiation.cc:407] Connection header received
0808 02:48:58.365899 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:58.365906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:58.366047 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:58.366137 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:58.366935 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.368237 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.370565 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.371321 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.371430 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.373278 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:58.373290 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:58.373307 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:58.374083 (+ 776us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:58.374142 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:58.374170 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:58.374261 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:58.374863 (+ 602us) server_negotiation.cc:299] Negotiation successful
0808 02:48:58.374984 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:48:58.377031 23626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:58.382009 24091 ts_tablet_manager.cc:1918] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:58.382200 24091 log.cc:1196] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/a36162a331ed4ddca20cab8c590a4f4f
I20260808 02:48:58.382532 24095 ts_tablet_manager.cc:1905] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:48:58.382735 24091 ts_tablet_manager.cc:1939] T a36162a331ed4ddca20cab8c590a4f4f P 2b3568c18e164f26ac3ee6fdcc4f9849: Deleting consensus metadata
I20260808 02:48:58.384599 23332 catalog_manager.cc:4841] TS 2b3568c18e164f26ac3ee6fdcc4f9849 (127.25.137.195:36213): tablet a36162a331ed4ddca20cab8c590a4f4f (table test-table [id=71edc29663af452e98a81be8d8979c05]) successfully deleted
I20260808 02:48:58.392002 24095 ts_tablet_manager.cc:1918] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:48:58.392192 24095 log.cc:1196] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/a36162a331ed4ddca20cab8c590a4f4f
I20260808 02:48:58.392839 24095 ts_tablet_manager.cc:1939] T a36162a331ed4ddca20cab8c590a4f4f P e019bc58cf7e4da8a410e6d71b2e347c: Deleting consensus metadata
I20260808 02:48:58.394870 23332 catalog_manager.cc:4841] TS e019bc58cf7e4da8a410e6d71b2e347c (127.25.137.194:36859): tablet a36162a331ed4ddca20cab8c590a4f4f (table test-table [id=71edc29663af452e98a81be8d8979c05]) successfully deleted
I20260808 02:48:58.470575 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 23443
W20260808 02:48:58.492728 23336 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.137.193:42171 (error 108)
I20260808 02:48:58.492902 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 23637
I20260808 02:48:58.494967 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.494413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42171 (local address 127.0.0.1:34874)
0808 02:48:58.494665 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.494796 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260808 02:48:58.495210 23336 catalog_manager.cc:4584] TS 82d9c7c069f54705ad396117d518c67b (127.25.137.193:42171): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a36162a331ed4ddca20cab8c590a4f4f: Network error: Client connection negotiation failed: client connection to 127.25.137.193:42171: connect: Connection refused (error 111)
I20260808 02:48:58.500350 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.493900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.25.137.194:36859)
0808 02:48:58.494118 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:58.494125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:58.494148 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:48:58.494367 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:58.494373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:58.494484 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:58.494594 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:58.495072 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.495939 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.497261 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.497700 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.497814 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.498867 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:58.498878 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:58.498892 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:58.499527 (+ 635us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:58.499575 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:58.499599 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:58.499692 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:58.499907 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 02:48:58.500010 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:48:58.501166 23821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:58.549058 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.548457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42171 (local address 127.0.0.1:34890)
0808 02:48:58.548716 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.548830 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 02:48:58.550216 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.549793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42171 (local address 127.0.0.1:34902)
0808 02:48:58.549956 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.550061 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 02:48:58.610868 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.610341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42171 (local address 127.0.0.1:34912)
0808 02:48:58.610557 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.610655 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 02:48:58.611941 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.611552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42171 (local address 127.0.0.1:34920)
0808 02:48:58.611706 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.611781 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 02:48:58.614914 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 23637
I20260808 02:48:58.642318 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 23835
I20260808 02:48:58.648869 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.643128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.25.137.195:36213)
0808 02:48:58.643292 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:48:58.643314 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 02:48:58.643363 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:48:58.643526 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:48:58.643530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:48:58.643614 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:48:58.643699 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:48:58.644138 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.644874 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.646185 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:48:58.646580 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:48:58.646670 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:48:58.647696 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:48:58.647704 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:48:58.647719 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:48:58.648285 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:48:58.648349 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:48:58.648370 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:48:58.648455 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:48:58.648538 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 02:48:58.648630 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:48:58.649498 24016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:58.692435 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.691883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42171 (local address 127.0.0.1:34926)
0808 02:48:58.692127 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.692245 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 02:48:58.693640 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.693188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42171 (local address 127.0.0.1:34940)
0808 02:48:58.693405 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.693499 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260808 02:48:58.738333 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 23835
I20260808 02:48:58.762681 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 23302
I20260808 02:48:58.763866 23394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:48:58.888136 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:48:58.887388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:42171 (local address 127.0.0.1:34942)
0808 02:48:58.887792 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 02:48:58.887912 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244}
I20260808 02:48:58.889093 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 23302
2026-08-08T02:48:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2882 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:48:59 dist-test-slave-k5rr krb5kdc[24105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:48:59 dist-test-slave-k5rr krb5kdc[24105](info): set up 2 sockets
Aug 08 02:48:59 dist-test-slave-k5rr krb5kdc[24105](info): commencing operation
krb5kdc: starting...
W20260808 02:49:00.158954 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.004s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:00 dist-test-slave-k5rr krb5kdc[24105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:49:00.474486 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:38095
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:38095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:00.811029 24127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:00.811317 24127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:00.811463 24127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:00.818466 24127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:00.818576 24127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:00.818655 24127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:00.818734 24127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:00.818802 24127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:00.828241 24127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:38095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:38095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:00.829535 24127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:00.830809 24127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:00.842434 24133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:00.842484 24132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:00.842523 24138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:00.843637 24127 server_base.cc:1034] running on GCE node
I20260808 02:49:00.844460 24127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:00.845909 24127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:00.847123 24127 hybrid_clock.cc:648] HybridClock initialized: now 1786157340847049 us; error 65 us; skew 500 ppm
I20260808 02:49:00.847463 24127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:00 dist-test-slave-k5rr krb5kdc[24105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157340, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:00.850551 24127 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:49:00.852039 24127 webserver.cc:460] Webserver started at http://127.25.137.254:45807/ using document root <none> and password file <none>
I20260808 02:49:00.852679 24127 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:00.852807 24127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:00.853142 24127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:00.856096 24127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "eccbaed6c5ff49de973efe45fcb447fc"
format_stamp: "Formatted at 2026-08-08 02:49:00 on dist-test-slave-k5rr"
server_key: "b2b1765de3b64d2acb0752dbf3ebc07a"
server_key_iv: "e4ff6b0bc0ffcffb232fd94a127dfb38"
server_key_version: "encryptionkey@0"
I20260808 02:49:00.856837 24127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "eccbaed6c5ff49de973efe45fcb447fc"
format_stamp: "Formatted at 2026-08-08 02:49:00 on dist-test-slave-k5rr"
server_key: "b2b1765de3b64d2acb0752dbf3ebc07a"
server_key_iv: "e4ff6b0bc0ffcffb232fd94a127dfb38"
server_key_version: "encryptionkey@0"
I20260808 02:49:00.861809 24127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 02:49:00.865113 24150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:00.866374 24127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:49:00.866532 24127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "eccbaed6c5ff49de973efe45fcb447fc"
format_stamp: "Formatted at 2026-08-08 02:49:00 on dist-test-slave-k5rr"
server_key: "b2b1765de3b64d2acb0752dbf3ebc07a"
server_key_iv: "e4ff6b0bc0ffcffb232fd94a127dfb38"
server_key_version: "encryptionkey@0"
I20260808 02:49:00.866705 24127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:00.881700 24127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:00.885819 24127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:00.886080 24127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:00.904167 24127 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:38095
I20260808 02:49:00.904176 24237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:38095 every 8 connection(s)
I20260808 02:49:00.905562 24127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:00.909339 24238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:00.914999 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 24127
I20260808 02:49:00.915184 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:00.915730 26151 external_mini_cluster.cc:1428] Setting key 989b5c77c99c6700e12d78f1d9c1ea50
I20260808 02:49:00.920598 24238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc: Bootstrap starting.
I20260808 02:49:00.924329 24238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:00.925590 24238 log.cc:824] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:00.929193 24238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc: No bootstrap required, opened a new log
Aug 08 02:49:00 dist-test-slave-k5rr krb5kdc[24105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:00.938935 24238 raft_consensus.cc:348] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccbaed6c5ff49de973efe45fcb447fc" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38095 } }
I20260808 02:49:00.939399 24238 raft_consensus.cc:374] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:00.939505 24238 raft_consensus.cc:729] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eccbaed6c5ff49de973efe45fcb447fc, State: Initialized, Role: FOLLOWER
I20260808 02:49:00.940068 24238 consensus_queue.cc:260] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccbaed6c5ff49de973efe45fcb447fc" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38095 } }
I20260808 02:49:00.940312 24238 raft_consensus.cc:388] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:00.940382 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:00.917711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52764 (local address 127.25.137.254:38095)
0808 02:49:00.918646 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:00.918658 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:00.918701 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:49:00.920424 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:00.920447 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:00.920869 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:00.921245 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:00.923171 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:00.924578 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:00.926436 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:00.927348 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:00.927486 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:00.932621 (+ 5135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:00.932630 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:00.932645 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:00.932688 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:00.935897 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:00.936829 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:00.936839 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:00.936852 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:00.937107 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:00.937517 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:00.937520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:00.937522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:00.938081 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:00.938303 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:00.938750 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 02:49:00.939110 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":427,"threads_started":1}
I20260808 02:49:00.940460 24238 raft_consensus.cc:482] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:00.940654 24238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:00.943517 24238 raft_consensus.cc:504] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccbaed6c5ff49de973efe45fcb447fc" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38095 } }
I20260808 02:49:00.943959 24238 leader_election.cc:302] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eccbaed6c5ff49de973efe45fcb447fc; no voters:
I20260808 02:49:00.944527 24238 leader_election.cc:288] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:00.944670 24249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:00.945459 24249 raft_consensus.cc:686] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [term 1 LEADER]: Becoming Leader. State: Replica: eccbaed6c5ff49de973efe45fcb447fc, State: Running, Role: LEADER
I20260808 02:49:00.946002 24249 consensus_queue.cc:237] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccbaed6c5ff49de973efe45fcb447fc" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38095 } }
I20260808 02:49:00.946507 24238 sys_catalog.cc:564] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:00.949915 24250 sys_catalog.cc:455] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eccbaed6c5ff49de973efe45fcb447fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccbaed6c5ff49de973efe45fcb447fc" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38095 } } }
I20260808 02:49:00.948915 24251 sys_catalog.cc:455] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader eccbaed6c5ff49de973efe45fcb447fc. Latest consensus state: current_term: 1 leader_uuid: "eccbaed6c5ff49de973efe45fcb447fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eccbaed6c5ff49de973efe45fcb447fc" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38095 } } }
I20260808 02:49:00.950493 24250 sys_catalog.cc:458] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:00.950493 24251 sys_catalog.cc:458] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:00.952020 24259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:00.957500 24259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:00.967404 24259 catalog_manager.cc:1347] Generated new cluster ID: f520af0000a241db8e97990c2096051b
I20260808 02:49:00.967543 24259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:00.983706 24259 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:00.985076 24259 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:00.992944 24259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc: Generated new TSK 0
I20260808 02:49:00.993814 24259 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
Entry for principal kudu/127.25.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab.
I20260808 02:49:01.084829 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38095
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:01.449904 24283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:01.450155 24283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:01.450246 24283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:01.457731 24283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:01.457895 24283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:01.458025 24283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:49:01.467813 24283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.193.keytab
--principal=kudu/127.25.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:01.468986 24283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:01.470366 24283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:01.484719 24295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:01.484803 24289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:01.484791 24293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:01.485745 24283 server_base.cc:1034] running on GCE node
I20260808 02:49:01.486289 24283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:01.487164 24283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:01.488366 24283 hybrid_clock.cc:648] HybridClock initialized: now 1786157341488325 us; error 44 us; skew 500 ppm
I20260808 02:49:01.488668 24283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:01 dist-test-slave-k5rr krb5kdc[24105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157341, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:01.491971 24283 init.cc:377] Logged in from keytab as kudu/127.25.137.193@KRBTEST.COM (short username kudu)
I20260808 02:49:01.493510 24283 webserver.cc:460] Webserver started at http://127.25.137.193:44173/ using document root <none> and password file <none>
I20260808 02:49:01.494158 24283 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:01.494282 24283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:01.494575 24283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:01.497434 24283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "5d61922a4ec54e5fa5715c62b1b53472"
format_stamp: "Formatted at 2026-08-08 02:49:01 on dist-test-slave-k5rr"
server_key: "c4638d42f6efa8e7c20e955dfca3b08c"
server_key_iv: "06dafd321e1953f2f27baefc2152562f"
server_key_version: "encryptionkey@0"
I20260808 02:49:01.498157 24283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "5d61922a4ec54e5fa5715c62b1b53472"
format_stamp: "Formatted at 2026-08-08 02:49:01 on dist-test-slave-k5rr"
server_key: "c4638d42f6efa8e7c20e955dfca3b08c"
server_key_iv: "06dafd321e1953f2f27baefc2152562f"
server_key_version: "encryptionkey@0"
I20260808 02:49:01.503074 24283 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:49:01.506371 24310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:01.507687 24283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 02:49:01.507910 24283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "5d61922a4ec54e5fa5715c62b1b53472"
format_stamp: "Formatted at 2026-08-08 02:49:01 on dist-test-slave-k5rr"
server_key: "c4638d42f6efa8e7c20e955dfca3b08c"
server_key_iv: "06dafd321e1953f2f27baefc2152562f"
server_key_version: "encryptionkey@0"
I20260808 02:49:01.508121 24283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:01.530612 24283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:01.534763 24283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:01.535039 24283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:01.536062 24283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:49:01.537456 24283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:49:01.537534 24283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:01.537658 24283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:49:01.537711 24283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:01.567992 24283 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:45005
I20260808 02:49:01.568032 24483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:45005 every 8 connection(s)
I20260808 02:49:01.569252 24283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:49:01.572994 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 24283
I20260808 02:49:01.573203 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:49:01.573810 26151 external_mini_cluster.cc:1428] Setting key ee49a768dcc582cde824bf77d6899aa6
Aug 08 02:49:01 dist-test-slave-k5rr krb5kdc[24105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157341, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.193@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:01.590344 24486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:01.571725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:48741 (local address 127.25.137.254:38095)
0808 02:49:01.572214 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:01.572218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:01.573220 (+ 1002us) server_negotiation.cc:407] Connection header received
0808 02:49:01.575419 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:01.575424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:01.575499 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:01.575608 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:01.577551 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:01.578265 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:01.580285 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:01.580667 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:01.580803 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:01.584614 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:01.584619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:01.584623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:01.584656 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:01.587158 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:01.587956 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:01.587960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:01.587962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:01.588082 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:01.588519 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:01.588523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:01.588525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:01.588820 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:01.588934 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:01.589725 (+ 791us) server_negotiation.cc:299] Negotiation successful
0808 02:49:01.589986 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":267,"threads_started":1}
I20260808 02:49:01.591261 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:01.572179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:38095 (local address 127.25.137.193:48741)
0808 02:49:01.572982 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:01.573079 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:01.575172 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:01.575899 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:01.575910 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:01.576625 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:01.577378 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:01.577402 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:01.578430 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:01.578434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:01.580137 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:01.580152 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:01.580891 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:01.580897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:01.581171 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:01.581841 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:01.581876 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:01.584382 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:01.587290 (+ 2908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:01.587299 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:01.587313 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:01.587839 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:01.588173 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:01.588178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:01.588181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:01.588408 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:01.588941 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:01.588948 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:01.589364 (+ 416us) client_negotiation.cc:769] Sending connection context
0808 02:49:01.589639 (+ 275us) client_negotiation.cc:240] Negotiation successful
0808 02:49:01.590192 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":259,"threads_started":1}
I20260808 02:49:01.592871 24484 heartbeater.cc:347] Connected to a master server at 127.25.137.254:38095
I20260808 02:49:01.593187 24484 heartbeater.cc:464] Registering TS with master...
I20260808 02:49:01.593910 24484 heartbeater.cc:511] Master 127.25.137.254:38095 requested a full tablet report, sending...
I20260808 02:49:01.595893 24177 ts_manager.cc:194] Registered new tserver with Master: 5d61922a4ec54e5fa5715c62b1b53472 (127.25.137.193:45005)
I20260808 02:49:01.597373 24177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.193@KRBTEST.COM'} at 127.25.137.193:48741
WARNING: no policy specified for kudu/127.25.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
Entry for principal kudu/127.25.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab.
I20260808 02:49:01.640599 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38095
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:01.972479 24490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:01.972754 24490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:01.972848 24490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:01.979962 24490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:01.980073 24490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:01.980195 24490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:49:01.989804 24490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.194.keytab
--principal=kudu/127.25.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:01.991086 24490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:01.992427 24490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:02.005038 24500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:02.005086 24501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:02.005064 24503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:02.006101 24490 server_base.cc:1034] running on GCE node
I20260808 02:49:02.006676 24490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:02.007570 24490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:02.008795 24490 hybrid_clock.cc:648] HybridClock initialized: now 1786157342008735 us; error 50 us; skew 500 ppm
I20260808 02:49:02.009171 24490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:02 dist-test-slave-k5rr krb5kdc[24105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157342, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:02.012327 24490 init.cc:377] Logged in from keytab as kudu/127.25.137.194@KRBTEST.COM (short username kudu)
I20260808 02:49:02.013921 24490 webserver.cc:460] Webserver started at http://127.25.137.194:33499/ using document root <none> and password file <none>
I20260808 02:49:02.014618 24490 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:02.014751 24490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:02.015087 24490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:02.018141 24490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "393783db762d4930a08f22c2e2f49794"
format_stamp: "Formatted at 2026-08-08 02:49:02 on dist-test-slave-k5rr"
server_key: "5ca8d96de753c11f8a1f0ebefa4575bc"
server_key_iv: "5c6e2ae500187111cd3e9b05295df9db"
server_key_version: "encryptionkey@0"
I20260808 02:49:02.018944 24490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "393783db762d4930a08f22c2e2f49794"
format_stamp: "Formatted at 2026-08-08 02:49:02 on dist-test-slave-k5rr"
server_key: "5ca8d96de753c11f8a1f0ebefa4575bc"
server_key_iv: "5c6e2ae500187111cd3e9b05295df9db"
server_key_version: "encryptionkey@0"
I20260808 02:49:02.024290 24490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:49:02.027778 24512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:02.029170 24490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:49:02.029335 24490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "393783db762d4930a08f22c2e2f49794"
format_stamp: "Formatted at 2026-08-08 02:49:02 on dist-test-slave-k5rr"
server_key: "5ca8d96de753c11f8a1f0ebefa4575bc"
server_key_iv: "5c6e2ae500187111cd3e9b05295df9db"
server_key_version: "encryptionkey@0"
I20260808 02:49:02.029532 24490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:02.055078 24490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:02.059372 24490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:02.059671 24490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:02.060703 24490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:49:02.062222 24490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:49:02.062311 24490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:02.062446 24490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:49:02.062510 24490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:02.095412 24490 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:42447
I20260808 02:49:02.095460 24688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:42447 every 8 connection(s)
I20260808 02:49:02.096844 24490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:49:02.100833 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 24490
I20260808 02:49:02.101044 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:49:02.101675 26151 external_mini_cluster.cc:1428] Setting key 7682f347cd79eb35a0352494d06f5f96
Aug 08 02:49:02 dist-test-slave-k5rr krb5kdc[24105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157342, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.194@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:02.118988 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.099620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:53115 (local address 127.25.137.254:38095)
0808 02:49:02.100054 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:02.100059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:02.101180 (+ 1121us) server_negotiation.cc:407] Connection header received
0808 02:49:02.103583 (+ 2403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:02.103588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:02.103667 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:02.103752 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:02.105303 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.106314 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.107867 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.108416 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.108549 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.112251 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:02.112258 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:02.112263 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:02.112303 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:02.115663 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:02.116483 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:02.116489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:02.116491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:02.116647 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:02.117179 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:02.117184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:02.117186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:02.117535 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:02.117677 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:02.118476 (+ 799us) server_negotiation.cc:299] Negotiation successful
0808 02:49:02.118709 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":215,"threads_started":1}
I20260808 02:49:02.120329 24695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.100150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:38095 (local address 127.25.137.194:53115)
0808 02:49:02.100971 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:02.101057 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:02.103262 (+ 2205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:02.103999 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:02.104007 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:02.104531 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:02.105129 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.105150 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.106481 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.106485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.107716 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.107728 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.108671 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.108677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.108903 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.109661 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:02.109686 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:02.112016 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:02.115848 (+ 3832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:02.115855 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:02.115868 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:02.116341 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:02.116800 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:02.116803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:02.116806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:02.117048 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:02.117678 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:02.117684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:02.118072 (+ 388us) client_negotiation.cc:769] Sending connection context
0808 02:49:02.118327 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 02:49:02.118813 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":299,"threads_started":1}
I20260808 02:49:02.122032 24690 heartbeater.cc:347] Connected to a master server at 127.25.137.254:38095
I20260808 02:49:02.122354 24690 heartbeater.cc:464] Registering TS with master...
I20260808 02:49:02.123072 24690 heartbeater.cc:511] Master 127.25.137.254:38095 requested a full tablet report, sending...
I20260808 02:49:02.124662 24177 ts_manager.cc:194] Registered new tserver with Master: 393783db762d4930a08f22c2e2f49794 (127.25.137.194:42447)
I20260808 02:49:02.125849 24177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.194@KRBTEST.COM'} at 127.25.137.194:53115
WARNING: no policy specified for kudu/127.25.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
Entry for principal kudu/127.25.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab.
I20260808 02:49:02.174446 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38095
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:02.508932 24698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:02.509224 24698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:02.509327 24698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:02.516732 24698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:02.516852 24698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:02.516973 24698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:49:02.526674 24698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.195.keytab
--principal=kudu/127.25.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:38095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:02.527997 24698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:02.529577 24698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:02.542485 24704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:02.542438 24705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:02.542510 24709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:02.543365 24698 server_base.cc:1034] running on GCE node
I20260808 02:49:02.543949 24698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:02.544968 24698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:02.546242 24698 hybrid_clock.cc:648] HybridClock initialized: now 1786157342546182 us; error 44 us; skew 500 ppm
I20260808 02:49:02.546598 24698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:02 dist-test-slave-k5rr krb5kdc[24105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157342, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:02.549933 24698 init.cc:377] Logged in from keytab as kudu/127.25.137.195@KRBTEST.COM (short username kudu)
I20260808 02:49:02.551400 24698 webserver.cc:460] Webserver started at http://127.25.137.195:42583/ using document root <none> and password file <none>
I20260808 02:49:02.552110 24698 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:02.552227 24698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:02.552510 24698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:02.555598 24698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "ff1eab0e0f4d4cc996623d25327911b7"
format_stamp: "Formatted at 2026-08-08 02:49:02 on dist-test-slave-k5rr"
server_key: "09b1e4c8a9319d92edc1b2457c9f1152"
server_key_iv: "c292ec5257979cc52705ae4fecbc4ff8"
server_key_version: "encryptionkey@0"
I20260808 02:49:02.556392 24698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "ff1eab0e0f4d4cc996623d25327911b7"
format_stamp: "Formatted at 2026-08-08 02:49:02 on dist-test-slave-k5rr"
server_key: "09b1e4c8a9319d92edc1b2457c9f1152"
server_key_iv: "c292ec5257979cc52705ae4fecbc4ff8"
server_key_version: "encryptionkey@0"
I20260808 02:49:02.561677 24698 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:49:02.564965 24723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:02.566241 24698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:49:02.566392 24698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "ff1eab0e0f4d4cc996623d25327911b7"
format_stamp: "Formatted at 2026-08-08 02:49:02 on dist-test-slave-k5rr"
server_key: "09b1e4c8a9319d92edc1b2457c9f1152"
server_key_iv: "c292ec5257979cc52705ae4fecbc4ff8"
server_key_version: "encryptionkey@0"
I20260808 02:49:02.566578 24698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:02.578312 24698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:02.582195 24698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:02.582454 24698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:02.583356 24698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:49:02.584627 24698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:49:02.584700 24698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:02.584823 24698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:49:02.584885 24698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:02.600106 24484 heartbeater.cc:503] Master 127.25.137.254:38095 was elected leader, sending a full tablet report...
I20260808 02:49:02.615742 24698 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:41045
I20260808 02:49:02.615793 24901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:41045 every 8 connection(s)
I20260808 02:49:02.616995 24698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:49:02.625710 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 24698
I20260808 02:49:02.625902 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:49:02.626468 26151 external_mini_cluster.cc:1428] Setting key 239bcee2831bb7b8c7eb986f56b53b78
Aug 08 02:49:02 dist-test-slave-k5rr krb5kdc[24105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157342, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.195@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:02.637892 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.619475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:56747 (local address 127.25.137.254:38095)
0808 02:49:02.619885 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:02.619890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:02.620785 (+ 895us) server_negotiation.cc:407] Connection header received
0808 02:49:02.622671 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:02.622677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:02.622767 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:02.622868 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:02.624338 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.625143 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.627268 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.627720 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.627858 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.631670 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:02.631675 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:02.631678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:02.631715 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:02.634210 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:02.635176 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:02.635183 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:02.635186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:02.635360 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:02.635868 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:02.635889 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:02.635893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:02.636264 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:02.636423 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:02.637291 (+ 868us) server_negotiation.cc:299] Negotiation successful
0808 02:49:02.637515 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":200,"threads_started":1}
I20260808 02:49:02.639166 24907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.619959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:38095 (local address 127.25.137.195:56747)
0808 02:49:02.620630 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:02.620674 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:02.622409 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:02.623106 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:02.623113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:02.623611 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:02.624164 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.624182 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.625316 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.625321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.627118 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.627132 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.627947 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.627955 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.628236 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.629063 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:02.629091 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:02.631449 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:02.634380 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:02.634389 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:02.634405 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:02.634998 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:02.635468 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:02.635472 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:02.635475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:02.635725 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:02.636396 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:02.636404 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:02.636852 (+ 448us) client_negotiation.cc:769] Sending connection context
0808 02:49:02.637196 (+ 344us) client_negotiation.cc:240] Negotiation successful
0808 02:49:02.637747 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":252,"threads_started":1}
I20260808 02:49:02.640645 24902 heartbeater.cc:347] Connected to a master server at 127.25.137.254:38095
I20260808 02:49:02.640930 24902 heartbeater.cc:464] Registering TS with master...
I20260808 02:49:02.641674 24902 heartbeater.cc:511] Master 127.25.137.254:38095 requested a full tablet report, sending...
I20260808 02:49:02.645390 24176 ts_manager.cc:194] Registered new tserver with Master: ff1eab0e0f4d4cc996623d25327911b7 (127.25.137.195:41045)
I20260808 02:49:02.646632 24176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.137.195@KRBTEST.COM'} at 127.25.137.195:56747
I20260808 02:49:02.646781 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:49:02.664018 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.651881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35900 (local address 127.25.137.254:38095)
0808 02:49:02.652102 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:02.652110 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:02.652457 (+ 347us) server_negotiation.cc:407] Connection header received
0808 02:49:02.652608 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:02.652611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:02.652684 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:02.652760 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:02.654222 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.654937 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.656187 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.656572 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.656660 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.659016 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:02.659022 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:02.659025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:02.659057 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:02.661609 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:02.662245 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:02.662249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:02.662251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:02.662376 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:02.662683 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:02.662686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:02.662688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:02.663056 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:02.663253 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:02.663514 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 02:49:02.663679 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:49:02.667785 24176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:49:02.674889 24176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:49:02.707983 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.688705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53268 (local address 127.25.137.194:42447)
0808 02:49:02.698825 (+ 10120us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:02.698835 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:02.698872 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:49:02.699410 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:02.699421 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:02.699723 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:02.699937 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:49:02.701021 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.702670 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.705202 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.706502 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.706665 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.706844 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:02.707460 (+ 616us) server_negotiation.cc:299] Negotiation successful
0808 02:49:02.707641 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9950,"thread_start_us":9811,"threads_started":1}
I20260808 02:49:02.708014 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.688066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:42447 (local address 127.0.0.1:53268)
0808 02:49:02.698762 (+ 10696us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:02.698785 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:02.698936 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:02.700277 (+ 1341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:02.700286 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:02.700336 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:49:02.700793 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.700805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.702850 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.702855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.704948 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.704960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.706980 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.706987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.707185 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.707244 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 02:49:02.707399 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 02:49:02.707600 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10547,"thread_start_us":10967,"threads_started":1}
I20260808 02:49:02.711138 24922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.687326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:41045 (local address 127.0.0.1:35114)
0808 02:49:02.688066 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:02.688102 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:02.688287 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:02.703682 (+ 15395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:02.703687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:02.703716 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:49:02.704151 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.704167 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.706241 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.706247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.708348 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.708364 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.710375 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.710381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.710558 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.710593 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:49:02.710678 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:49:02.710804 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":306,"threads_started":1}
I20260808 02:49:02.711814 24582 tablet_service.cc:1467] Processing CreateTablet for tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9 (DEFAULT_TABLE table=test-table [id=6c57cdaf89034385b4c8403d6396d8c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:49:02.713084 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.687528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35114 (local address 127.25.137.195:41045)
0808 02:49:02.702846 (+ 15318us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:02.702854 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:02.702889 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:49:02.702992 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:02.703000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:02.703253 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:02.704371 (+ 1118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:49:02.704691 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.706082 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.708514 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.709906 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.710095 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.710248 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:02.711544 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0808 02:49:02.711711 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15112,"thread_start_us":15881,"threads_started":1}
I20260808 02:49:02.713158 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.688389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45005 (local address 127.0.0.1:49958)
0808 02:49:02.702344 (+ 13955us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:02.702364 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:02.702485 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:02.704954 (+ 2469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:02.704959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:02.704985 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:49:02.705378 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.705390 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.707305 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.707309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.709629 (+ 2320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.709645 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.711875 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.711882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.712046 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.712080 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:49:02.712159 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 02:49:02.712276 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13807,"thread_start_us":14177,"threads_started":1}
I20260808 02:49:02.713452 24582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:02.713558 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.689355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49958 (local address 127.25.137.193:45005)
0808 02:49:02.704373 (+ 15018us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:02.704381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:02.704408 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:49:02.704506 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:02.704513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:02.704743 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:02.704959 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:49:02.705535 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.707172 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.709814 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.711641 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.711825 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.711972 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:02.712731 (+ 759us) server_negotiation.cc:299] Negotiation successful
0808 02:49:02.712906 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14853,"thread_start_us":14680,"threads_started":1}
I20260808 02:49:02.716707 24384 tablet_service.cc:1467] Processing CreateTablet for tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9 (DEFAULT_TABLE table=test-table [id=6c57cdaf89034385b4c8403d6396d8c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:49:02.716764 24792 tablet_service.cc:1467] Processing CreateTablet for tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9 (DEFAULT_TABLE table=test-table [id=6c57cdaf89034385b4c8403d6396d8c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:49:02.717931 24384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:02.718184 24792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:02.728184 24931 tablet_bootstrap.cc:492] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: Bootstrap starting.
I20260808 02:49:02.730099 24933 tablet_bootstrap.cc:492] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: Bootstrap starting.
I20260808 02:49:02.732019 24935 tablet_bootstrap.cc:492] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7: Bootstrap starting.
I20260808 02:49:02.732384 24931 tablet_bootstrap.cc:654] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:02.733664 24933 tablet_bootstrap.cc:654] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:02.733932 24931 log.cc:824] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:02.734736 24933 log.cc:824] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:02.735785 24935 tablet_bootstrap.cc:654] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:02.737035 24935 log.cc:824] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:02.737816 24931 tablet_bootstrap.cc:492] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: No bootstrap required, opened a new log
I20260808 02:49:02.737917 24933 tablet_bootstrap.cc:492] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: No bootstrap required, opened a new log
I20260808 02:49:02.738099 24931 ts_tablet_manager.cc:1397] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.001s
I20260808 02:49:02.738209 24933 ts_tablet_manager.cc:1397] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260808 02:49:02.740549 24935 tablet_bootstrap.cc:492] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7: No bootstrap required, opened a new log
I20260808 02:49:02.740794 24935 ts_tablet_manager.cc:1397] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 02:49:02.745783 24931 raft_consensus.cc:348] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1eab0e0f4d4cc996623d25327911b7" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41045 } } peers { permanent_uuid: "5d61922a4ec54e5fa5715c62b1b53472" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45005 } } peers { permanent_uuid: "393783db762d4930a08f22c2e2f49794" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42447 } }
I20260808 02:49:02.746178 24931 raft_consensus.cc:374] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:02.746279 24931 raft_consensus.cc:729] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393783db762d4930a08f22c2e2f49794, State: Initialized, Role: FOLLOWER
I20260808 02:49:02.746881 24931 consensus_queue.cc:260] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1eab0e0f4d4cc996623d25327911b7" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41045 } } peers { permanent_uuid: "5d61922a4ec54e5fa5715c62b1b53472" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45005 } } peers { permanent_uuid: "393783db762d4930a08f22c2e2f49794" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42447 } }
I20260808 02:49:02.748106 24690 heartbeater.cc:503] Master 127.25.137.254:38095 was elected leader, sending a full tablet report...
I20260808 02:49:02.748274 24933 raft_consensus.cc:348] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1eab0e0f4d4cc996623d25327911b7" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41045 } } peers { permanent_uuid: "5d61922a4ec54e5fa5715c62b1b53472" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45005 } } peers { permanent_uuid: "393783db762d4930a08f22c2e2f49794" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42447 } }
I20260808 02:49:02.748585 24931 ts_tablet_manager.cc:1428] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: Time spent starting tablet: real 0.010s user 0.004s sys 0.006s
I20260808 02:49:02.748685 24933 raft_consensus.cc:374] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:02.748785 24933 raft_consensus.cc:729] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d61922a4ec54e5fa5715c62b1b53472, State: Initialized, Role: FOLLOWER
I20260808 02:49:02.749423 24933 consensus_queue.cc:260] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1eab0e0f4d4cc996623d25327911b7" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41045 } } peers { permanent_uuid: "5d61922a4ec54e5fa5715c62b1b53472" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45005 } } peers { permanent_uuid: "393783db762d4930a08f22c2e2f49794" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42447 } }
I20260808 02:49:02.751065 24933 ts_tablet_manager.cc:1428] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20260808 02:49:02.752065 24935 raft_consensus.cc:348] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1eab0e0f4d4cc996623d25327911b7" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41045 } } peers { permanent_uuid: "5d61922a4ec54e5fa5715c62b1b53472" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45005 } } peers { permanent_uuid: "393783db762d4930a08f22c2e2f49794" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42447 } }
I20260808 02:49:02.752569 24935 raft_consensus.cc:374] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:02.752672 24935 raft_consensus.cc:729] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1eab0e0f4d4cc996623d25327911b7, State: Initialized, Role: FOLLOWER
I20260808 02:49:02.753441 24935 consensus_queue.cc:260] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1eab0e0f4d4cc996623d25327911b7" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41045 } } peers { permanent_uuid: "5d61922a4ec54e5fa5715c62b1b53472" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45005 } } peers { permanent_uuid: "393783db762d4930a08f22c2e2f49794" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42447 } }
I20260808 02:49:02.754498 24902 heartbeater.cc:503] Master 127.25.137.254:38095 was elected leader, sending a full tablet report...
I20260808 02:49:02.755102 24935 ts_tablet_manager.cc:1428] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7: Time spent starting tablet: real 0.014s user 0.003s sys 0.011s
I20260808 02:49:02.789826 24943 raft_consensus.cc:482] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:49:02.790140 24943 raft_consensus.cc:504] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1eab0e0f4d4cc996623d25327911b7" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41045 } } peers { permanent_uuid: "5d61922a4ec54e5fa5715c62b1b53472" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45005 } } peers { permanent_uuid: "393783db762d4930a08f22c2e2f49794" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42447 } }
I20260808 02:49:02.792689 24943 leader_election.cc:288] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d61922a4ec54e5fa5715c62b1b53472 (127.25.137.193:45005), 393783db762d4930a08f22c2e2f49794 (127.25.137.194:42447)
I20260808 02:49:02.797468 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.792493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:33727 (local address 127.25.137.194:42447)
0808 02:49:02.792697 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:02.792703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:02.792722 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:49:02.792947 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:02.792954 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:02.793064 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:02.793157 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:49:02.793552 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.794265 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.795821 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.796605 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.796749 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.796834 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:02.797092 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 02:49:02.797218 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 02:49:02.797468 24907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.792254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:42447 (local address 127.25.137.195:33727)
0808 02:49:02.792567 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:02.792586 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:02.792727 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:02.793201 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:02.793206 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:02.793233 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:49:02.793418 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.793426 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.794395 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.794401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.795652 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.795668 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.796751 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.796756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.796925 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.796949 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:49:02.797067 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 02:49:02.797194 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260808 02:49:02.798579 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.792410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:45005 (local address 127.25.137.195:36089)
0808 02:49:02.793311 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:02.793325 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:02.793456 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:02.793764 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:02.793769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:02.793792 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:49:02.794140 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.794150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.795112 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.795115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.796446 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:02.796457 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.797864 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.797869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:02.797980 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.798002 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:49:02.798083 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 02:49:02.798200 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":31,"thread_start_us":555,"threads_started":1}
I20260808 02:49:02.798580 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.792373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:36089 (local address 127.25.137.193:45005)
0808 02:49:02.792567 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:02.792573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:02.793423 (+ 850us) server_negotiation.cc:407] Connection header received
0808 02:49:02.793588 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:02.793592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:02.793659 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:02.793734 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:49:02.794289 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.794966 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.796601 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.797422 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.797568 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.797643 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:02.798132 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 02:49:02.798313 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:49:02.798547 24615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69e6f9e7f5f45e89f6b5d3eaae2eaf9" candidate_uuid: "ff1eab0e0f4d4cc996623d25327911b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393783db762d4930a08f22c2e2f49794" is_pre_election: true
I20260808 02:49:02.799084 24615 raft_consensus.cc:2454] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff1eab0e0f4d4cc996623d25327911b7 in term 0.
I20260808 02:49:02.799520 24413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69e6f9e7f5f45e89f6b5d3eaae2eaf9" candidate_uuid: "ff1eab0e0f4d4cc996623d25327911b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d61922a4ec54e5fa5715c62b1b53472" is_pre_election: true
I20260808 02:49:02.799718 24725 leader_election.cc:302] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 393783db762d4930a08f22c2e2f49794, ff1eab0e0f4d4cc996623d25327911b7; no voters:
I20260808 02:49:02.799985 24413 raft_consensus.cc:2454] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff1eab0e0f4d4cc996623d25327911b7 in term 0.
I20260808 02:49:02.800091 24943 raft_consensus.cc:2781] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:49:02.800231 24943 raft_consensus.cc:482] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:49:02.800321 24943 raft_consensus.cc:3037] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:02.802959 24943 raft_consensus.cc:504] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1eab0e0f4d4cc996623d25327911b7" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41045 } } peers { permanent_uuid: "5d61922a4ec54e5fa5715c62b1b53472" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45005 } } peers { permanent_uuid: "393783db762d4930a08f22c2e2f49794" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42447 } }
I20260808 02:49:02.803563 24943 leader_election.cc:288] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [CANDIDATE]: Term 1 election: Requested vote from peers 5d61922a4ec54e5fa5715c62b1b53472 (127.25.137.193:45005), 393783db762d4930a08f22c2e2f49794 (127.25.137.194:42447)
I20260808 02:49:02.803920 24413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69e6f9e7f5f45e89f6b5d3eaae2eaf9" candidate_uuid: "ff1eab0e0f4d4cc996623d25327911b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d61922a4ec54e5fa5715c62b1b53472"
I20260808 02:49:02.804072 24615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69e6f9e7f5f45e89f6b5d3eaae2eaf9" candidate_uuid: "ff1eab0e0f4d4cc996623d25327911b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393783db762d4930a08f22c2e2f49794"
I20260808 02:49:02.804145 24413 raft_consensus.cc:3037] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:02.804287 24615 raft_consensus.cc:3037] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:02.807447 24615 raft_consensus.cc:2454] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff1eab0e0f4d4cc996623d25327911b7 in term 1.
I20260808 02:49:02.807447 24413 raft_consensus.cc:2454] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff1eab0e0f4d4cc996623d25327911b7 in term 1.
I20260808 02:49:02.807885 24725 leader_election.cc:302] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 393783db762d4930a08f22c2e2f49794, ff1eab0e0f4d4cc996623d25327911b7; no voters:
I20260808 02:49:02.808192 24943 raft_consensus.cc:2781] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:02.808669 24943 raft_consensus.cc:686] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 1 LEADER]: Becoming Leader. State: Replica: ff1eab0e0f4d4cc996623d25327911b7, State: Running, Role: LEADER
I20260808 02:49:02.809186 24943 consensus_queue.cc:237] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1eab0e0f4d4cc996623d25327911b7" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41045 } } peers { permanent_uuid: "5d61922a4ec54e5fa5715c62b1b53472" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45005 } } peers { permanent_uuid: "393783db762d4930a08f22c2e2f49794" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42447 } }
I20260808 02:49:02.813961 24177 catalog_manager.cc:5495] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff1eab0e0f4d4cc996623d25327911b7 (127.25.137.195). New cstate: current_term: 1 leader_uuid: "ff1eab0e0f4d4cc996623d25327911b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1eab0e0f4d4cc996623d25327911b7" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 41045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d61922a4ec54e5fa5715c62b1b53472" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "393783db762d4930a08f22c2e2f49794" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42447 } health_report { overall_health: UNKNOWN } } }
W20260808 02:49:02.821542 24485 tablet.cc:2369] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:49:02.837095 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.831954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35124 (local address 127.25.137.195:41045)
0808 02:49:02.832148 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:02.832152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:02.832170 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:49:02.832317 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:02.832320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:02.832410 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:02.832527 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:02.832951 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.833714 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.835133 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.835518 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.835609 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.836062 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:02.836532 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:02.836645 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:02.836728 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 02:49:02.836793 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:49:02.844732 24615 raft_consensus.cc:1261] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794 [term 1 FOLLOWER]: Refusing update from remote peer ff1eab0e0f4d4cc996623d25327911b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:49:02.844738 24413 raft_consensus.cc:1261] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472 [term 1 FOLLOWER]: Refusing update from remote peer ff1eab0e0f4d4cc996623d25327911b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:49:02.845714 24949 consensus_queue.cc:1035] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "393783db762d4930a08f22c2e2f49794" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 42447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:49:02.846046 24943 consensus_queue.cc:1035] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d61922a4ec54e5fa5715c62b1b53472" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 45005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:49:02.848860 24691 tablet.cc:2369] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:49:02.857192 24963 mvcc.cc:205] Tried to move back new op lower bound from 7316100476282884096 to 7316100476150509568. Current Snapshot: MvccSnapshot[applied={T|T < 7316100476282884096}]
I20260808 02:49:02.870793 24177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:49:02.871153 24177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:49:02.874737 24177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eccbaed6c5ff49de973efe45fcb447fc: Sending DeleteTablet for 3 replicas of tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9
I20260808 02:49:02.875885 24384 tablet_service.cc:1514] Processing DeleteTablet for tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:49:02 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:49958
I20260808 02:49:02.875953 24582 tablet_service.cc:1514] Processing DeleteTablet for tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:49:02 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:53268
I20260808 02:49:02.876570 24792 tablet_service.cc:1514] Processing DeleteTablet for tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:49:02 UTC) from {username='kudu', principal='kudu/127.25.137.254@KRBTEST.COM'} at 127.0.0.1:35114
I20260808 02:49:02.877043 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 24283
Aug 08 02:49:02 dist-test-slave-k5rr krb5kdc[24105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.193@KRBTEST.COM
I20260808 02:49:02.900712 24976 tablet_replica.cc:330] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7: stopping tablet replica
I20260808 02:49:02.901479 24976 raft_consensus.cc:2229] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:49:02.902566 24976 raft_consensus.cc:2258] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:49:02.903425 24974 tablet_replica.cc:330] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: stopping tablet replica
I20260808 02:49:02.904209 24974 raft_consensus.cc:2229] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:49:02.905089 24974 raft_consensus.cc:2258] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:49:02.906514 24976 ts_tablet_manager.cc:1905] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:49:02.908929 24974 ts_tablet_manager.cc:1905] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:49:02.911846 24975 tablet_replica.cc:330] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: stopping tablet replica
I20260808 02:49:02.912267 24975 raft_consensus.cc:2229] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:49:02.912786 24975 raft_consensus.cc:2258] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:49:02.916090 24976 ts_tablet_manager.cc:1918] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:49:02.916255 24976 log.cc:1196] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/e69e6f9e7f5f45e89f6b5d3eaae2eaf9
I20260808 02:49:02.916787 24976 ts_tablet_manager.cc:1939] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P ff1eab0e0f4d4cc996623d25327911b7: Deleting consensus metadata
I20260808 02:49:02.918512 24155 catalog_manager.cc:4841] TS ff1eab0e0f4d4cc996623d25327911b7 (127.25.137.195:41045): tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9 (table test-table [id=6c57cdaf89034385b4c8403d6396d8c8]) successfully deleted
I20260808 02:49:02.920186 24975 ts_tablet_manager.cc:1905] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:49:02.920750 24974 ts_tablet_manager.cc:1918] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:49:02.920997 24974 log.cc:1196] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-0/wal/wals/e69e6f9e7f5f45e89f6b5d3eaae2eaf9
I20260808 02:49:02.921636 24974 ts_tablet_manager.cc:1939] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 5d61922a4ec54e5fa5715c62b1b53472: Deleting consensus metadata
I20260808 02:49:02.922699 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:02.878353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49960 (local address 127.25.137.193:45005)
0808 02:49:02.878524 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:02.878530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:02.888855 (+ 10325us) server_negotiation.cc:407] Connection header received
0808 02:49:02.889338 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:02.889344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:02.889439 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:02.889510 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:02.891369 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.892553 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.894804 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:02.895492 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:02.895589 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:02.900950 (+ 5361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:02.900969 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:02.900988 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:02.901050 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:02.904927 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:02.919767 (+ 14840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:02.919777 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:02.919780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:02.919989 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:02.920525 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:02.920530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:02.920533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:02.921137 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:02.921462 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:02.922052 (+ 590us) server_negotiation.cc:299] Negotiation successful
0808 02:49:02.922296 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:49:02.923887 24155 catalog_manager.cc:4841] TS 5d61922a4ec54e5fa5715c62b1b53472 (127.25.137.193:45005): tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9 (table test-table [id=6c57cdaf89034385b4c8403d6396d8c8]) successfully deleted
I20260808 02:49:02.924194 24472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:02.927098 24975 ts_tablet_manager.cc:1918] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:49:02.927274 24975 log.cc:1196] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/e69e6f9e7f5f45e89f6b5d3eaae2eaf9
I20260808 02:49:02.927815 24975 ts_tablet_manager.cc:1939] T e69e6f9e7f5f45e89f6b5d3eaae2eaf9 P 393783db762d4930a08f22c2e2f49794: Deleting consensus metadata
I20260808 02:49:02.929684 24153 catalog_manager.cc:4841] TS 393783db762d4930a08f22c2e2f49794 (127.25.137.194:42447): tablet e69e6f9e7f5f45e89f6b5d3eaae2eaf9 (table test-table [id=6c57cdaf89034385b4c8403d6396d8c8]) successfully deleted
I20260808 02:49:03.021302 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 24283
I20260808 02:49:03.041245 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 24490
Aug 08 02:49:03 dist-test-slave-k5rr krb5kdc[24105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.194@KRBTEST.COM
I20260808 02:49:03.056624 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:03.042113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53282 (local address 127.25.137.194:42447)
0808 02:49:03.042326 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:03.042332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:03.042353 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:49:03.042417 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:03.042420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:03.042492 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:03.042575 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:03.043872 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:03.044620 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:03.046111 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:03.046606 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:03.046737 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:03.050909 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:03.050924 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:03.050939 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:03.050977 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:03.053917 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:03.054567 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:03.054574 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:03.054577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:03.054742 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:03.055110 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:03.055114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:03.055116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:03.055505 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:03.055779 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:03.056098 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 02:49:03.056320 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:49:03.057329 24681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:03.153230 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 24490
I20260808 02:49:03.180474 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 24698
Aug 08 02:49:03 dist-test-slave-k5rr krb5kdc[24105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.195@KRBTEST.COM
I20260808 02:49:03.199616 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:03.181284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35126 (local address 127.25.137.195:41045)
0808 02:49:03.181743 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:03.181748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:03.181764 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:49:03.181830 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:03.181837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:03.181902 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:03.181981 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:03.183952 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:03.184783 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:03.186849 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:03.187601 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:03.187743 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:03.193512 (+ 5769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:03.193527 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:03.193543 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:03.193599 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:03.196559 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:03.197516 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:03.197523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:03.197526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:03.197673 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:03.198129 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:03.198132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:03.198134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:03.198510 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:03.198743 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:03.199115 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 02:49:03.199291 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329}
I20260808 02:49:03.200285 24890 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:03.325685 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 24698
I20260808 02:49:03.350859 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 24127
I20260808 02:49:03.351917 24230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:03.463845 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 24127
2026-08-08T02:49:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4569 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T02:49:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:03Z Disabled control of system clock
I20260808 02:49:03.564468 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:37885
--webserver_interface=127.25.137.254
--webserver_port=0
--builtin_ntp_servers=127.25.137.212:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:37885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:49:03.902372 24991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:03.902655 24991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:03.902752 24991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:03.909967 24991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:03.910072 24991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:03.910120 24991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:03.910175 24991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:03.910218 24991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:03.919368 24991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:37885
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:37885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:03.920585 24991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:03.922358 24991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:03.933730 25001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:03.933758 25005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:03.934101 25000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:03.934880 24991 server_base.cc:1034] running on GCE node
I20260808 02:49:03.935539 24991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:03.936839 24991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:03.938030 24991 hybrid_clock.cc:648] HybridClock initialized: now 1786157343937958 us; error 69 us; skew 500 ppm
I20260808 02:49:03.938354 24991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:49:03.940536 24991 webserver.cc:460] Webserver started at http://127.25.137.254:35377/ using document root <none> and password file <none>
I20260808 02:49:03.941243 24991 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:03.941342 24991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:03.941640 24991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:03.944416 24991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "a64dc812dca145f4be71e644228e44b8"
format_stamp: "Formatted at 2026-08-08 02:49:03 on dist-test-slave-k5rr"
server_key: "b1b9c7d9230376757f4a2236a868cb5f"
server_key_iv: "9324985fb5a4370a8e30ff25fb2c59d5"
server_key_version: "encryptionkey@0"
I20260808 02:49:03.945451 24991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "a64dc812dca145f4be71e644228e44b8"
format_stamp: "Formatted at 2026-08-08 02:49:03 on dist-test-slave-k5rr"
server_key: "b1b9c7d9230376757f4a2236a868cb5f"
server_key_iv: "9324985fb5a4370a8e30ff25fb2c59d5"
server_key_version: "encryptionkey@0"
I20260808 02:49:03.950841 24991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:49:03.954437 25016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:03.955739 24991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:49:03.955950 24991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "a64dc812dca145f4be71e644228e44b8"
format_stamp: "Formatted at 2026-08-08 02:49:03 on dist-test-slave-k5rr"
server_key: "b1b9c7d9230376757f4a2236a868cb5f"
server_key_iv: "9324985fb5a4370a8e30ff25fb2c59d5"
server_key_version: "encryptionkey@0"
I20260808 02:49:03.956143 24991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:03.981154 24991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:03.982019 24991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:03.982292 24991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:04.000188 24991 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:37885
I20260808 02:49:04.000208 25091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:37885 every 8 connection(s)
I20260808 02:49:04.001511 24991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:04.004034 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 24991
I20260808 02:49:04.004220 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:04.004817 26151 external_mini_cluster.cc:1428] Setting key 9b93edf309295c5f5560081c8242e175
I20260808 02:49:04.005836 25092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:04.021271 25092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8: Bootstrap starting.
I20260808 02:49:04.023954 25095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:04.008182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47136 (local address 127.25.137.254:37885)
0808 02:49:04.009447 (+ 1265us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:04.009465 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:04.009680 (+ 215us) server_negotiation.cc:407] Connection header received
0808 02:49:04.012143 (+ 2463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:04.012180 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:04.012846 (+ 666us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:04.013357 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:04.014583 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:04.016478 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:04.018516 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:04.019586 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:04.019755 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:04.020964 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:04.020975 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:04.020992 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:04.021806 (+ 814us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:04.021919 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:04.021946 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:04.022061 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:04.022330 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 02:49:04.022641 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":399,"threads_started":1}
I20260808 02:49:04.025199 25092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:04.026235 25092 log.cc:824] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:04.028889 25092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8: No bootstrap required, opened a new log
I20260808 02:49:04.035981 25092 raft_consensus.cc:348] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64dc812dca145f4be71e644228e44b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37885 } }
I20260808 02:49:04.036347 25092 raft_consensus.cc:374] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:04.036438 25092 raft_consensus.cc:729] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a64dc812dca145f4be71e644228e44b8, State: Initialized, Role: FOLLOWER
I20260808 02:49:04.037043 25092 consensus_queue.cc:260] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64dc812dca145f4be71e644228e44b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37885 } }
I20260808 02:49:04.037293 25092 raft_consensus.cc:388] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:04.037418 25092 raft_consensus.cc:482] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:04.037577 25092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:04.039685 25092 raft_consensus.cc:504] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64dc812dca145f4be71e644228e44b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37885 } }
I20260808 02:49:04.040167 25092 leader_election.cc:302] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a64dc812dca145f4be71e644228e44b8; no voters:
I20260808 02:49:04.040779 25092 leader_election.cc:288] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:04.040925 25102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:04.041716 25102 raft_consensus.cc:686] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [term 1 LEADER]: Becoming Leader. State: Replica: a64dc812dca145f4be71e644228e44b8, State: Running, Role: LEADER
I20260808 02:49:04.042243 25102 consensus_queue.cc:237] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64dc812dca145f4be71e644228e44b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37885 } }
I20260808 02:49:04.042876 25092 sys_catalog.cc:564] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:04.044945 25106 sys_catalog.cc:455] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a64dc812dca145f4be71e644228e44b8. Latest consensus state: current_term: 1 leader_uuid: "a64dc812dca145f4be71e644228e44b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64dc812dca145f4be71e644228e44b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37885 } } }
I20260808 02:49:04.045132 25105 sys_catalog.cc:455] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a64dc812dca145f4be71e644228e44b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64dc812dca145f4be71e644228e44b8" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37885 } } }
I20260808 02:49:04.045358 25106 sys_catalog.cc:458] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:04.045445 25105 sys_catalog.cc:458] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:04.047190 25116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:04.051862 25116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:04.059942 25116 catalog_manager.cc:1347] Generated new cluster ID: 69874fbc9a9f4fda826b1ed548af0ed4
I20260808 02:49:04.060060 25116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:04.071607 25116 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:04.072916 25116 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:04.084448 25116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8: Generated new TSK 0
I20260808 02:49:04.085321 25116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:04.094920 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.193:0
--local_ip_for_outbound_sockets=127.25.137.193
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37885
--builtin_ntp_servers=127.25.137.212:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:49:04.431010 25138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:04.431298 25138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:04.431428 25138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:04.438690 25138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:04.438808 25138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:04.438954 25138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.193
I20260808 02:49:04.447955 25138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.137.193
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.193
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:04.449254 25138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:04.451012 25138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:04.463464 25146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:04.463491 25147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:04.463651 25149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:04.464354 25138 server_base.cc:1034] running on GCE node
I20260808 02:49:04.465031 25138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:04.465996 25138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:04.467200 25138 hybrid_clock.cc:648] HybridClock initialized: now 1786157344467151 us; error 43 us; skew 500 ppm
I20260808 02:49:04.467510 25138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:49:04.469686 25138 webserver.cc:460] Webserver started at http://127.25.137.193:43911/ using document root <none> and password file <none>
I20260808 02:49:04.470331 25138 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:04.470433 25138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:04.470729 25138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:04.473551 25138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/data/instance:
uuid: "749c0b9d2b3e499ca9e17246da27579f"
format_stamp: "Formatted at 2026-08-08 02:49:04 on dist-test-slave-k5rr"
server_key: "176a27a8611a864c2bbf9d7f25443ec4"
server_key_iv: "01f0f6dcf1f941addf570a2960b718fd"
server_key_version: "encryptionkey@0"
I20260808 02:49:04.474318 25138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance:
uuid: "749c0b9d2b3e499ca9e17246da27579f"
format_stamp: "Formatted at 2026-08-08 02:49:04 on dist-test-slave-k5rr"
server_key: "176a27a8611a864c2bbf9d7f25443ec4"
server_key_iv: "01f0f6dcf1f941addf570a2960b718fd"
server_key_version: "encryptionkey@0"
I20260808 02:49:04.479141 25138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:49:04.482518 25161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:04.483736 25138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 02:49:04.483886 25138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/wal
uuid: "749c0b9d2b3e499ca9e17246da27579f"
format_stamp: "Formatted at 2026-08-08 02:49:04 on dist-test-slave-k5rr"
server_key: "176a27a8611a864c2bbf9d7f25443ec4"
server_key_iv: "01f0f6dcf1f941addf570a2960b718fd"
server_key_version: "encryptionkey@0"
I20260808 02:49:04.484073 25138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:04.515308 25138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:04.516151 25138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:04.516449 25138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:04.517467 25138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:49:04.518924 25138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:49:04.519026 25138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:04.519134 25138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:49:04.519207 25138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:04.549750 25138 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.193:43451
I20260808 02:49:04.549831 25343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.193:43451 every 8 connection(s)
I20260808 02:49:04.551007 25138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/data/info.pb
I20260808 02:49:04.557190 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 25138
I20260808 02:49:04.557349 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-0/wal/instance
I20260808 02:49:04.557760 26151 external_mini_cluster.cc:1428] Setting key 3d400d824b30ac660195b7550f6e14ee
I20260808 02:49:04.560770 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.194:0
--local_ip_for_outbound_sockets=127.25.137.194
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37885
--builtin_ntp_servers=127.25.137.212:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:49:04.563621 25348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:04.553422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:54271 (local address 127.25.137.254:37885)
0808 02:49:04.553837 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:04.553842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:04.554968 (+ 1126us) server_negotiation.cc:407] Connection header received
0808 02:49:04.556717 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:04.556723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:04.556825 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:04.556940 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:04.558072 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:04.559016 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:04.560914 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:04.561469 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:04.561594 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:04.562853 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:04.562860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:04.562864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:04.562884 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:04.562930 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:04.562944 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:04.563030 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:04.563202 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 02:49:04.563301 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":214,"threads_started":1}
I20260808 02:49:04.564520 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:04.553918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:37885 (local address 127.25.137.193:54271)
0808 02:49:04.554813 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:04.554857 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:04.556456 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:04.557192 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:04.557200 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:04.557294 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:04.557906 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:04.557925 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:04.559195 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:04.559201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:04.560759 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:04.560772 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:04.561696 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:04.561701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:04.561898 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:04.562546 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:49:04.562548 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:04.562587 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:49:04.562662 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:04.563033 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:04.563040 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:04.563045 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:49:04.563176 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 02:49:04.563472 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":260,"threads_started":1}
I20260808 02:49:04.565897 25345 heartbeater.cc:347] Connected to a master server at 127.25.137.254:37885
I20260808 02:49:04.566205 25345 heartbeater.cc:464] Registering TS with master...
I20260808 02:49:04.567049 25345 heartbeater.cc:511] Master 127.25.137.254:37885 requested a full tablet report, sending...
I20260808 02:49:04.569089 25040 ts_manager.cc:194] Registered new tserver with Master: 749c0b9d2b3e499ca9e17246da27579f (127.25.137.193:43451)
I20260808 02:49:04.570581 25040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.193:54271
W20260808 02:49:04.892905 25353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:04.893230 25353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:04.893364 25353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:04.900668 25353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:04.900769 25353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:04.900913 25353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.194
I20260808 02:49:04.910064 25353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.137.194
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.194
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:04.911320 25353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:04.913182 25353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:04.925874 25364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:04.925926 25366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:04.926121 25368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:04.926959 25353 server_base.cc:1034] running on GCE node
I20260808 02:49:04.927524 25353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:04.928431 25353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:04.929631 25353 hybrid_clock.cc:648] HybridClock initialized: now 1786157344929572 us; error 46 us; skew 500 ppm
I20260808 02:49:04.929963 25353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:49:04.932185 25353 webserver.cc:460] Webserver started at http://127.25.137.194:35519/ using document root <none> and password file <none>
I20260808 02:49:04.932890 25353 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:04.932999 25353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:04.933334 25353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:04.936148 25353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/data/instance:
uuid: "3ee9066ddbf249e19cc95b43f4f371d5"
format_stamp: "Formatted at 2026-08-08 02:49:04 on dist-test-slave-k5rr"
server_key: "69f24ea5852e514ae66f53c836246614"
server_key_iv: "f27918500bb699e98500077c8ea52146"
server_key_version: "encryptionkey@0"
I20260808 02:49:04.936903 25353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance:
uuid: "3ee9066ddbf249e19cc95b43f4f371d5"
format_stamp: "Formatted at 2026-08-08 02:49:04 on dist-test-slave-k5rr"
server_key: "69f24ea5852e514ae66f53c836246614"
server_key_iv: "f27918500bb699e98500077c8ea52146"
server_key_version: "encryptionkey@0"
I20260808 02:49:04.942309 25353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260808 02:49:04.946211 25378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:04.947718 25353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:49:04.947880 25353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/wal
uuid: "3ee9066ddbf249e19cc95b43f4f371d5"
format_stamp: "Formatted at 2026-08-08 02:49:04 on dist-test-slave-k5rr"
server_key: "69f24ea5852e514ae66f53c836246614"
server_key_iv: "f27918500bb699e98500077c8ea52146"
server_key_version: "encryptionkey@0"
I20260808 02:49:04.948123 25353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:04.959998 25353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:04.960861 25353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:04.961151 25353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:04.962117 25353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:49:04.963543 25353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:49:04.963631 25353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:04.963714 25353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:49:04.963778 25353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:04.994521 25353 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.194:39947
I20260808 02:49:04.994575 25557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.194:39947 every 8 connection(s)
I20260808 02:49:04.995905 25353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/data/info.pb
I20260808 02:49:05.003202 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 25353
I20260808 02:49:05.003403 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/wal/instance
I20260808 02:49:05.003855 26151 external_mini_cluster.cc:1428] Setting key 43d8648faf047b60cc4579e21c0e4c3e
I20260808 02:49:05.007170 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.137.195:0
--local_ip_for_outbound_sockets=127.25.137.195
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37885
--builtin_ntp_servers=127.25.137.212:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:49:05.010787 25348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:04.998472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.194:40821 (local address 127.25.137.254:37885)
0808 02:49:04.998672 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:04.998676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:05.000303 (+ 1627us) server_negotiation.cc:407] Connection header received
0808 02:49:05.002731 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:05.002735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:05.002860 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:05.002984 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:05.004299 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.005241 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.007489 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.008028 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.008126 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.009953 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:05.009959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:05.009964 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:05.009988 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:05.010033 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:05.010048 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:05.010133 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:05.010358 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 02:49:05.010463 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:49:05.012101 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:04.999007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:37885 (local address 127.25.137.194:40821)
0808 02:49:05.000115 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:05.000173 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:05.002427 (+ 2254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:05.003226 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:05.003237 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:05.003357 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:05.004110 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.004133 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.005408 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.005412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.007311 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.007331 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.008461 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.008467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.008759 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.009576 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:49:05.009580 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:05.009643 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:49:05.009745 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:05.010145 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:05.010155 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:05.010162 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:49:05.010356 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 02:49:05.010762 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":323,"threads_started":1}
I20260808 02:49:05.013640 25562 heartbeater.cc:347] Connected to a master server at 127.25.137.254:37885
I20260808 02:49:05.013957 25562 heartbeater.cc:464] Registering TS with master...
I20260808 02:49:05.014802 25562 heartbeater.cc:511] Master 127.25.137.254:37885 requested a full tablet report, sending...
I20260808 02:49:05.016517 25040 ts_manager.cc:194] Registered new tserver with Master: 3ee9066ddbf249e19cc95b43f4f371d5 (127.25.137.194:39947)
I20260808 02:49:05.017686 25040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.194:40821
W20260808 02:49:05.345539 25565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:05.345796 25565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:05.345882 25565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:05.353018 25565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:05.353120 25565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:05.353230 25565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.137.195
I20260808 02:49:05.362319 25565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.137.195
--webserver_port=0
--tserver_master_addrs=127.25.137.254:37885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.137.195
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:05.363529 25565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:05.365325 25565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:05.377719 25575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:05.377761 25578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:05.377846 25576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:05.379127 25565 server_base.cc:1034] running on GCE node
I20260808 02:49:05.379686 25565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:05.380604 25565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:05.381800 25565 hybrid_clock.cc:648] HybridClock initialized: now 1786157345381740 us; error 55 us; skew 500 ppm
I20260808 02:49:05.382122 25565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:49:05.384272 25565 webserver.cc:460] Webserver started at http://127.25.137.195:45897/ using document root <none> and password file <none>
I20260808 02:49:05.384953 25565 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:05.385088 25565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:05.385394 25565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:05.388257 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/data/instance:
uuid: "e40b32a80bff483097bc04821c9de1af"
format_stamp: "Formatted at 2026-08-08 02:49:05 on dist-test-slave-k5rr"
server_key: "d9f6095facc966f8b4070774b7d1863f"
server_key_iv: "7d3bbdd0f66a8f7216824e729c273950"
server_key_version: "encryptionkey@0"
I20260808 02:49:05.389123 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance:
uuid: "e40b32a80bff483097bc04821c9de1af"
format_stamp: "Formatted at 2026-08-08 02:49:05 on dist-test-slave-k5rr"
server_key: "d9f6095facc966f8b4070774b7d1863f"
server_key_iv: "7d3bbdd0f66a8f7216824e729c273950"
server_key_version: "encryptionkey@0"
I20260808 02:49:05.394126 25565 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:49:05.397863 25591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:05.399276 25565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:49:05.399448 25565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/wal
uuid: "e40b32a80bff483097bc04821c9de1af"
format_stamp: "Formatted at 2026-08-08 02:49:05 on dist-test-slave-k5rr"
server_key: "d9f6095facc966f8b4070774b7d1863f"
server_key_iv: "7d3bbdd0f66a8f7216824e729c273950"
server_key_version: "encryptionkey@0"
I20260808 02:49:05.399665 25565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:05.416913 25565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:05.417798 25565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:05.418092 25565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:05.419073 25565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:49:05.420565 25565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:49:05.420665 25565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:05.420768 25565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:49:05.420837 25565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:05.454227 25565 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.195:38451
I20260808 02:49:05.454290 25775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.195:38451 every 8 connection(s)
I20260808 02:49:05.455612 25565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/data/info.pb
I20260808 02:49:05.461107 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 25565
I20260808 02:49:05.461263 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/wal/instance
I20260808 02:49:05.461697 26151 external_mini_cluster.cc:1428] Setting key f3dc237586e34cd29e2d2d5e9dfbac15
I20260808 02:49:05.468533 25348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.458139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.195:51837 (local address 127.25.137.254:37885)
0808 02:49:05.458316 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:05.458321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:05.459688 (+ 1367us) server_negotiation.cc:407] Connection header received
0808 02:49:05.461854 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:05.461860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:05.461952 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:05.462071 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:05.463471 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.464343 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.466098 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.466560 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.466675 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.467878 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:05.467883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:05.467886 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:05.467904 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:05.467937 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:05.467949 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:05.468018 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:05.468182 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 02:49:05.468260 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:49:05.469574 25778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.458556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:37885 (local address 127.25.137.195:51837)
0808 02:49:05.459501 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:05.459565 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:05.461554 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:05.462327 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:05.462340 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:05.462462 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:05.463282 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.463307 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.464543 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.464548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.465935 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.465948 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.466747 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.466752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.466945 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.467559 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:49:05.467562 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:05.467602 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:49:05.467685 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:05.468024 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:05.468030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:05.468036 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:49:05.468170 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 02:49:05.468484 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":277,"threads_started":1}
I20260808 02:49:05.470867 25776 heartbeater.cc:347] Connected to a master server at 127.25.137.254:37885
I20260808 02:49:05.471156 25776 heartbeater.cc:464] Registering TS with master...
I20260808 02:49:05.471917 25776 heartbeater.cc:511] Master 127.25.137.254:37885 requested a full tablet report, sending...
I20260808 02:49:05.473385 25040 ts_manager.cc:194] Registered new tserver with Master: e40b32a80bff483097bc04821c9de1af (127.25.137.195:38451)
I20260808 02:49:05.474187 25040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.137.195:51837
I20260808 02:49:05.477052 26151 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:49:05.488868 25348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.482489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.25.137.254:37885)
0808 02:49:05.482761 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:05.482766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:05.483058 (+ 292us) server_negotiation.cc:407] Connection header received
0808 02:49:05.483263 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:05.483268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:05.483354 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:05.483437 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:05.484040 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.484859 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.486600 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.487017 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.487128 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.488168 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:05.488173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:05.488176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:05.488193 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:05.488235 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:05.488247 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:05.488329 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:05.488454 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 02:49:05.488570 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 02:49:05.492834 25040 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:49:05.499684 25040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:49:05.523559 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.511768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:39947 (local address 127.0.0.1:55654)
0808 02:49:05.513139 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:05.513172 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:05.513374 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:05.514528 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:05.514535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:05.514567 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:49:05.515161 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.515176 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.517613 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.517623 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.520107 (+ 2484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.520127 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.522316 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.522324 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.522590 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.522659 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 02:49:05.522784 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 02:49:05.523005 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1180,"mutex_wait_us":1,"spinlock_wait_cycles":2304,"thread_start_us":842,"threads_started":1}
I20260808 02:49:05.524183 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.512396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55654 (local address 127.25.137.194:39947)
0808 02:49:05.513860 (+ 1464us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:05.513869 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:05.513900 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:49:05.514013 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:05.514022 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:05.514278 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:05.515425 (+ 1147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:49:05.515671 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.517442 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.520406 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.522027 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.523325 (+ 1298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.523558 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:05.523639 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 02:49:05.523833 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1302,"thread_start_us":1186,"threads_started":1}
I20260808 02:49:05.524507 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.511262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:43451 (local address 127.0.0.1:44730)
0808 02:49:05.513125 (+ 1863us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:05.513171 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:05.513378 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:05.514466 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:05.514475 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:05.514508 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:49:05.515046 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.515060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.517163 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.517169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.519822 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.519840 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.522676 (+ 2836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.522683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.522871 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.522908 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:49:05.522997 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 02:49:05.523136 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1597,"mutex_wait_us":51,"spinlock_wait_cycles":4352,"thread_start_us":1158,"threads_started":1}
I20260808 02:49:05.525298 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.511597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44730 (local address 127.25.137.193:43451)
0808 02:49:05.512263 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:05.512274 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:05.513575 (+ 1301us) server_negotiation.cc:407] Connection header received
0808 02:49:05.513683 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:05.513691 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:05.513948 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:05.514164 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:49:05.515270 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.516961 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.520046 (+ 3085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.522103 (+ 2057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.522270 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.522525 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:05.524725 (+ 2200us) server_negotiation.cc:299] Negotiation successful
0808 02:49:05.524919 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":326,"threads_started":1}
I20260808 02:49:05.529374 25238 tablet_service.cc:1467] Processing CreateTablet for tablet 872d0be2128843e09e34b85ca96fb0ff (DEFAULT_TABLE table=test-table [id=a889f12e190544e3ad1e6598df918a62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:49:05.529374 25461 tablet_service.cc:1467] Processing CreateTablet for tablet 872d0be2128843e09e34b85ca96fb0ff (DEFAULT_TABLE table=test-table [id=a889f12e190544e3ad1e6598df918a62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:49:05.530881 25238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 872d0be2128843e09e34b85ca96fb0ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:05.530881 25461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 872d0be2128843e09e34b85ca96fb0ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:05.531937 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.513308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:38451 (local address 127.0.0.1:47692)
0808 02:49:05.523890 (+ 10582us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:05.523911 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:05.524015 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:05.526097 (+ 2082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:05.526102 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:05.526128 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:49:05.526284 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.526295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.527782 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.527786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.529646 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.529660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.531080 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.531088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.531310 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.531347 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:49:05.531432 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:49:05.531594 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10482,"thread_start_us":12548,"threads_started":1}
I20260808 02:49:05.532069 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.513588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47692 (local address 127.25.137.195:38451)
0808 02:49:05.525435 (+ 11847us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:05.525447 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:05.525490 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:49:05.525608 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:05.525618 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:05.525893 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:05.526100 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:49:05.526429 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.527621 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.529813 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.530820 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.531033 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.531278 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:05.531519 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 02:49:05.531685 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11617,"thread_start_us":11432,"threads_started":1}
I20260808 02:49:05.535311 25663 tablet_service.cc:1467] Processing CreateTablet for tablet 872d0be2128843e09e34b85ca96fb0ff (DEFAULT_TABLE table=test-table [id=a889f12e190544e3ad1e6598df918a62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:49:05.536809 25663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 872d0be2128843e09e34b85ca96fb0ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:05.546088 25805 tablet_bootstrap.cc:492] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f: Bootstrap starting.
I20260808 02:49:05.548220 25806 tablet_bootstrap.cc:492] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: Bootstrap starting.
I20260808 02:49:05.550765 25805 tablet_bootstrap.cc:654] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:05.552405 25805 log.cc:824] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:05.552935 25806 tablet_bootstrap.cc:654] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:05.554255 25806 log.cc:824] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:05.557221 25807 tablet_bootstrap.cc:492] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: Bootstrap starting.
I20260808 02:49:05.561933 25805 tablet_bootstrap.cc:492] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f: No bootstrap required, opened a new log
I20260808 02:49:05.562276 25805 ts_tablet_manager.cc:1397] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260808 02:49:05.562371 25807 tablet_bootstrap.cc:654] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:05.562963 25806 tablet_bootstrap.cc:492] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: No bootstrap required, opened a new log
I20260808 02:49:05.563259 25806 ts_tablet_manager.cc:1397] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260808 02:49:05.563884 25807 log.cc:824] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:05.567067 25807 tablet_bootstrap.cc:492] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: No bootstrap required, opened a new log
I20260808 02:49:05.567358 25807 ts_tablet_manager.cc:1397] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 02:49:05.570475 25805 raft_consensus.cc:348] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749c0b9d2b3e499ca9e17246da27579f" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43451 } } peers { permanent_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39947 } } peers { permanent_uuid: "e40b32a80bff483097bc04821c9de1af" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38451 } }
I20260808 02:49:05.570823 25805 raft_consensus.cc:374] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:05.570926 25805 raft_consensus.cc:729] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 749c0b9d2b3e499ca9e17246da27579f, State: Initialized, Role: FOLLOWER
I20260808 02:49:05.571521 25805 consensus_queue.cc:260] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749c0b9d2b3e499ca9e17246da27579f" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43451 } } peers { permanent_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39947 } } peers { permanent_uuid: "e40b32a80bff483097bc04821c9de1af" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38451 } }
I20260808 02:49:05.573863 25345 heartbeater.cc:503] Master 127.25.137.254:37885 was elected leader, sending a full tablet report...
I20260808 02:49:05.573810 25806 raft_consensus.cc:348] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749c0b9d2b3e499ca9e17246da27579f" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43451 } } peers { permanent_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39947 } } peers { permanent_uuid: "e40b32a80bff483097bc04821c9de1af" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38451 } }
I20260808 02:49:05.574265 25806 raft_consensus.cc:374] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:05.574361 25806 raft_consensus.cc:729] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ee9066ddbf249e19cc95b43f4f371d5, State: Initialized, Role: FOLLOWER
I20260808 02:49:05.575100 25806 consensus_queue.cc:260] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749c0b9d2b3e499ca9e17246da27579f" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43451 } } peers { permanent_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39947 } } peers { permanent_uuid: "e40b32a80bff483097bc04821c9de1af" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38451 } }
I20260808 02:49:05.578372 25807 raft_consensus.cc:348] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749c0b9d2b3e499ca9e17246da27579f" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43451 } } peers { permanent_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39947 } } peers { permanent_uuid: "e40b32a80bff483097bc04821c9de1af" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38451 } }
I20260808 02:49:05.578951 25807 raft_consensus.cc:374] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:05.579104 25807 raft_consensus.cc:729] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40b32a80bff483097bc04821c9de1af, State: Initialized, Role: FOLLOWER
I20260808 02:49:05.579869 25807 consensus_queue.cc:260] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749c0b9d2b3e499ca9e17246da27579f" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43451 } } peers { permanent_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39947 } } peers { permanent_uuid: "e40b32a80bff483097bc04821c9de1af" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38451 } }
I20260808 02:49:05.581166 25776 heartbeater.cc:503] Master 127.25.137.254:37885 was elected leader, sending a full tablet report...
I20260808 02:49:05.582026 25805 ts_tablet_manager.cc:1428] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f: Time spent starting tablet: real 0.020s user 0.014s sys 0.005s
I20260808 02:49:05.585059 25807 ts_tablet_manager.cc:1428] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: Time spent starting tablet: real 0.018s user 0.012s sys 0.004s
I20260808 02:49:05.585769 25562 heartbeater.cc:503] Master 127.25.137.254:37885 was elected leader, sending a full tablet report...
I20260808 02:49:05.586403 25806 ts_tablet_manager.cc:1428] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: Time spent starting tablet: real 0.023s user 0.015s sys 0.009s
I20260808 02:49:05.665208 25813 raft_consensus.cc:482] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:49:05.665577 25813 raft_consensus.cc:504] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749c0b9d2b3e499ca9e17246da27579f" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43451 } } peers { permanent_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39947 } } peers { permanent_uuid: "e40b32a80bff483097bc04821c9de1af" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38451 } }
I20260808 02:49:05.667344 25813 leader_election.cc:288] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ee9066ddbf249e19cc95b43f4f371d5 (127.25.137.194:39947), e40b32a80bff483097bc04821c9de1af (127.25.137.195:38451)
I20260808 02:49:05.675292 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.667813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:38709 (local address 127.25.137.194:39947)
0808 02:49:05.668006 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:05.668011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:05.668501 (+ 490us) server_negotiation.cc:407] Connection header received
0808 02:49:05.668724 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:05.668731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:05.668840 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:05.668959 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:49:05.669713 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.670670 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.673351 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.674289 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.674413 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.674463 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:05.674899 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 02:49:05.675035 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:49:05.675319 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.668868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.137.193:40299 (local address 127.25.137.195:38451)
0808 02:49:05.669114 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:05.669121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:05.669343 (+ 222us) server_negotiation.cc:407] Connection header received
0808 02:49:05.669528 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:05.669531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:05.669591 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:05.669691 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:49:05.670196 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.671152 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.673503 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.674458 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.674621 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.674689 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:05.674973 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 02:49:05.675104 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 02:49:05.675418 25816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.667721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.194:39947 (local address 127.25.137.193:38709)
0808 02:49:05.668394 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:05.668414 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:05.668563 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:05.669154 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:05.669159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:05.669187 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:49:05.669568 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.669578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.670821 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.670825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.673139 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.673156 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.674493 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.674500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.674721 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.674755 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:49:05.674840 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:49:05.674948 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":105,"thread_start_us":359,"threads_started":1}
I20260808 02:49:05.675508 25818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.668531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.195:38451 (local address 127.25.137.193:40299)
0808 02:49:05.669228 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:05.669254 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:05.669409 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:05.669717 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:05.669720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:05.669743 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:49:05.670039 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.670047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.671294 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.671299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.673334 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:05.673347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.674638 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.674642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:05.674807 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.674838 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:49:05.674927 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 02:49:05.675044 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":40,"spinlock_wait_cycles":10624,"thread_start_us":336,"threads_started":1}
I20260808 02:49:05.676434 25694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872d0be2128843e09e34b85ca96fb0ff" candidate_uuid: "749c0b9d2b3e499ca9e17246da27579f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e40b32a80bff483097bc04821c9de1af" is_pre_election: true
I20260808 02:49:05.676524 25489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872d0be2128843e09e34b85ca96fb0ff" candidate_uuid: "749c0b9d2b3e499ca9e17246da27579f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" is_pre_election: true
I20260808 02:49:05.676934 25694 raft_consensus.cc:2454] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 749c0b9d2b3e499ca9e17246da27579f in term 0.
I20260808 02:49:05.677034 25489 raft_consensus.cc:2454] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 749c0b9d2b3e499ca9e17246da27579f in term 0.
I20260808 02:49:05.677582 25168 leader_election.cc:302] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 749c0b9d2b3e499ca9e17246da27579f, e40b32a80bff483097bc04821c9de1af; no voters:
I20260808 02:49:05.677927 25813 raft_consensus.cc:2781] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:49:05.678051 25813 raft_consensus.cc:482] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:49:05.678155 25813 raft_consensus.cc:3037] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:05.680426 25813 raft_consensus.cc:504] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749c0b9d2b3e499ca9e17246da27579f" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43451 } } peers { permanent_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39947 } } peers { permanent_uuid: "e40b32a80bff483097bc04821c9de1af" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38451 } }
I20260808 02:49:05.681043 25813 leader_election.cc:288] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [CANDIDATE]: Term 1 election: Requested vote from peers 3ee9066ddbf249e19cc95b43f4f371d5 (127.25.137.194:39947), e40b32a80bff483097bc04821c9de1af (127.25.137.195:38451)
I20260808 02:49:05.681577 25489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872d0be2128843e09e34b85ca96fb0ff" candidate_uuid: "749c0b9d2b3e499ca9e17246da27579f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ee9066ddbf249e19cc95b43f4f371d5"
I20260808 02:49:05.681675 25694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872d0be2128843e09e34b85ca96fb0ff" candidate_uuid: "749c0b9d2b3e499ca9e17246da27579f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e40b32a80bff483097bc04821c9de1af"
I20260808 02:49:05.681767 25489 raft_consensus.cc:3037] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:05.681882 25694 raft_consensus.cc:3037] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:05.684044 25694 raft_consensus.cc:2454] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 749c0b9d2b3e499ca9e17246da27579f in term 1.
I20260808 02:49:05.684048 25489 raft_consensus.cc:2454] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 749c0b9d2b3e499ca9e17246da27579f in term 1.
I20260808 02:49:05.684401 25168 leader_election.cc:302] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ee9066ddbf249e19cc95b43f4f371d5, 749c0b9d2b3e499ca9e17246da27579f; no voters:
I20260808 02:49:05.684707 25813 raft_consensus.cc:2781] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:05.685225 25813 raft_consensus.cc:686] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 1 LEADER]: Becoming Leader. State: Replica: 749c0b9d2b3e499ca9e17246da27579f, State: Running, Role: LEADER
I20260808 02:49:05.685693 25813 consensus_queue.cc:237] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749c0b9d2b3e499ca9e17246da27579f" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43451 } } peers { permanent_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39947 } } peers { permanent_uuid: "e40b32a80bff483097bc04821c9de1af" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38451 } }
I20260808 02:49:05.690115 25039 catalog_manager.cc:5495] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f reported cstate change: term changed from 0 to 1, leader changed from <none> to 749c0b9d2b3e499ca9e17246da27579f (127.25.137.193). New cstate: current_term: 1 leader_uuid: "749c0b9d2b3e499ca9e17246da27579f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "749c0b9d2b3e499ca9e17246da27579f" member_type: VOTER last_known_addr { host: "127.25.137.193" port: 43451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e40b32a80bff483097bc04821c9de1af" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38451 } health_report { overall_health: UNKNOWN } } }
W20260808 02:49:05.707319 25777 tablet.cc:2369] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:49:05.723336 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.717947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.25.137.193:43451)
0808 02:49:05.718153 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:05.718159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:05.718178 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:49:05.718421 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:05.718425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:05.718541 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:05.718673 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:05.719067 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.719781 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.721183 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.721634 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.721750 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.722226 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:05.722730 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:05.722877 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:05.722980 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 02:49:05.723074 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:49:05.731870 25694 raft_consensus.cc:1261] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af [term 1 FOLLOWER]: Refusing update from remote peer 749c0b9d2b3e499ca9e17246da27579f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:49:05.731992 25489 raft_consensus.cc:1261] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5 [term 1 FOLLOWER]: Refusing update from remote peer 749c0b9d2b3e499ca9e17246da27579f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:49:05.732851 25813 consensus_queue.cc:1035] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e40b32a80bff483097bc04821c9de1af" member_type: VOTER last_known_addr { host: "127.25.137.195" port: 38451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:49:05.733249 25819 consensus_queue.cc:1035] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ee9066ddbf249e19cc95b43f4f371d5" member_type: VOTER last_known_addr { host: "127.25.137.194" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:49:05.748062 25563 tablet.cc:2369] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:49:05.750550 25823 mvcc.cc:205] Tried to move back new op lower bound from 7316100488107720704 to 7316100487932125184. Current Snapshot: MvccSnapshot[applied={T|T < 7316100488107720704}]
I20260808 02:49:05.759078 25040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:49:05.759330 25040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:49:05.762920 25040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a64dc812dca145f4be71e644228e44b8: Sending DeleteTablet for 3 replicas of tablet 872d0be2128843e09e34b85ca96fb0ff
I20260808 02:49:05.764016 25238 tablet_service.cc:1514] Processing DeleteTablet for tablet 872d0be2128843e09e34b85ca96fb0ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:49:05 UTC) from {username='slave'} at 127.0.0.1:44730
I20260808 02:49:05.764070 25461 tablet_service.cc:1514] Processing DeleteTablet for tablet 872d0be2128843e09e34b85ca96fb0ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:49:05 UTC) from {username='slave'} at 127.0.0.1:55654
I20260808 02:49:05.764272 25663 tablet_service.cc:1514] Processing DeleteTablet for tablet 872d0be2128843e09e34b85ca96fb0ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:49:05 UTC) from {username='slave'} at 127.0.0.1:47692
I20260808 02:49:05.765913 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 25138
I20260808 02:49:05.773538 25840 tablet_replica.cc:330] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: stopping tablet replica
I20260808 02:49:05.773728 25839 tablet_replica.cc:330] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: stopping tablet replica
I20260808 02:49:05.774019 25840 raft_consensus.cc:2229] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:49:05.774065 25839 raft_consensus.cc:2229] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:49:05.774461 25839 raft_consensus.cc:2258] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:49:05.774511 25840 raft_consensus.cc:2258] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:49:05.776826 25839 ts_tablet_manager.cc:1905] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:49:05.777326 25840 ts_tablet_manager.cc:1905] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:49:05.784319 25839 ts_tablet_manager.cc:1918] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:49:05.784565 25839 log.cc:1196] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-1/wal/wals/872d0be2128843e09e34b85ca96fb0ff
I20260808 02:49:05.785223 25839 ts_tablet_manager.cc:1939] T 872d0be2128843e09e34b85ca96fb0ff P 3ee9066ddbf249e19cc95b43f4f371d5: Deleting consensus metadata
I20260808 02:49:05.787155 25023 catalog_manager.cc:4841] TS 3ee9066ddbf249e19cc95b43f4f371d5 (127.25.137.194:39947): tablet 872d0be2128843e09e34b85ca96fb0ff (table test-table [id=a889f12e190544e3ad1e6598df918a62]) successfully deleted
I20260808 02:49:05.789577 25838 tablet_replica.cc:330] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f: stopping tablet replica
I20260808 02:49:05.789978 25838 raft_consensus.cc:2229] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:49:05.790612 25838 raft_consensus.cc:2258] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:49:05.792201 25840 ts_tablet_manager.cc:1918] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:49:05.792459 25840 log.cc:1196] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: Deleting WAL directory at /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157019169307-26151-0/minicluster-data/ts-2/wal/wals/872d0be2128843e09e34b85ca96fb0ff
I20260808 02:49:05.792951 25838 ts_tablet_manager.cc:1905] T 872d0be2128843e09e34b85ca96fb0ff P 749c0b9d2b3e499ca9e17246da27579f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:49:05.793265 25840 ts_tablet_manager.cc:1939] T 872d0be2128843e09e34b85ca96fb0ff P e40b32a80bff483097bc04821c9de1af: Deleting consensus metadata
I20260808 02:49:05.795147 25023 catalog_manager.cc:4841] TS e40b32a80bff483097bc04821c9de1af (127.25.137.195:38451): tablet 872d0be2128843e09e34b85ca96fb0ff (table test-table [id=a889f12e190544e3ad1e6598df918a62]) successfully deleted
I20260808 02:49:05.796386 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.766949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.25.137.193:43451)
0808 02:49:05.767386 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:05.767392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:05.775389 (+ 7997us) server_negotiation.cc:407] Connection header received
0808 02:49:05.785418 (+ 10029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:05.785428 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:05.785567 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:05.785660 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:05.786526 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.787974 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.792069 (+ 4095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.792760 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.792892 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.794408 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:05.794419 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:05.794437 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:05.795250 (+ 813us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:05.795327 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:05.795355 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:05.795453 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:05.795732 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 02:49:05.795865 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302}
I20260808 02:49:05.797224 25338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:05.893805 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 25138
W20260808 02:49:05.916363 25020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.137.193:43451 (error 108)
I20260808 02:49:05.916504 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 25353
I20260808 02:49:05.918231 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.917792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:43451 (local address 127.0.0.1:44768)
0808 02:49:05.917977 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:05.918068 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:43451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 02:49:05.918484 25020 catalog_manager.cc:4584] TS 749c0b9d2b3e499ca9e17246da27579f (127.25.137.193:43451): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 872d0be2128843e09e34b85ca96fb0ff: Network error: Client connection negotiation failed: client connection to 127.25.137.193:43451: connect: Connection refused (error 111)
I20260808 02:49:05.923689 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.917310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55664 (local address 127.25.137.194:39947)
0808 02:49:05.917496 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:05.917502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:05.917593 (+ 91us) server_negotiation.cc:407] Connection header received
0808 02:49:05.917792 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:05.917797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:05.917891 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:05.917975 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:05.918441 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.919361 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.920642 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:05.921141 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:05.921247 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:05.922269 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:05.922278 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:05.922312 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:05.922890 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:05.922944 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:05.922965 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:05.923054 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:05.923219 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 02:49:05.923341 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:49:05.924393 25550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:05.972208 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.971706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:43451 (local address 127.0.0.1:44782)
0808 02:49:05.971956 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:05.972061 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:43451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260808 02:49:05.973309 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:05.972864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:43451 (local address 127.0.0.1:44786)
0808 02:49:05.973053 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:05.973142 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:43451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 02:49:06.012502 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 25353
I20260808 02:49:06.033845 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:06.033369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:43451 (local address 127.0.0.1:44788)
0808 02:49:06.033586 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:06.033682 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:43451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 02:49:06.034917 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:06.034529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:43451 (local address 127.0.0.1:44802)
0808 02:49:06.034680 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:06.034753 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:43451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 02:49:06.038000 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 25565
I20260808 02:49:06.044333 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:06.038689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47698 (local address 127.25.137.195:38451)
0808 02:49:06.038866 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:06.038871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:06.038891 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:49:06.039012 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:06.039015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:06.039099 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:06.039178 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:06.039533 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:06.040248 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:06.041605 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:06.041983 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:06.042076 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:06.043011 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:06.043018 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:06.043032 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:06.043674 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:06.043721 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:06.043741 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:06.043832 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:06.043952 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 02:49:06.044043 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:49:06.044947 25765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:06.116297 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:06.115682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:43451 (local address 127.0.0.1:44818)
0808 02:49:06.115993 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:06.116114 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:43451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 02:49:06.117537 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:06.116994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.193:43451 (local address 127.0.0.1:44828)
0808 02:49:06.117255 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:06.117363 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.193:43451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260808 02:49:06.133136 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 25565
I20260808 02:49:06.157228 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 24991
I20260808 02:49:06.158381 25081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:06.267729 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 24991
2026-08-08T02:49:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2801 ms)
[----------] 6 tests from EncryptionPolicyTest (23326 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:06 dist-test-slave-k5rr krb5kdc[25856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:06 dist-test-slave-k5rr krb5kdc[25856](info): set up 2 sockets
Aug 08 02:49:06 dist-test-slave-k5rr krb5kdc[25856](info): commencing operation
krb5kdc: starting...
W20260808 02:49:07.516465 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:07 dist-test-slave-k5rr krb5kdc[25856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:07.841948 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45999
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:08.177167 25883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:08.177471 25883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:08.177619 25883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:08.184769 25883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:08.184904 25883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:08.184986 25883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:08.185088 25883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:08.185153 25883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:08.194998 25883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45999
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:08.196309 25883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:08.197629 25883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:08.209268 25889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:08.209350 25891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:08.209338 25896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:08.210199 25883 server_base.cc:1034] running on GCE node
I20260808 02:49:08.210915 25883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:08.212292 25883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:08.213498 25883 hybrid_clock.cc:648] HybridClock initialized: now 1786157348213448 us; error 38 us; skew 500 ppm
I20260808 02:49:08.213871 25883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:08 dist-test-slave-k5rr krb5kdc[25856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157348, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:08.222030 25883 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:08.223474 25883 webserver.cc:460] Webserver started at http://127.0.0.1:36817/ using document root <none> and password file <none>
I20260808 02:49:08.224136 25883 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:08.224246 25883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:08.224509 25883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:08.227545 25883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "824b425bc306441289621c4e13ee8243"
format_stamp: "Formatted at 2026-08-08 02:49:08 on dist-test-slave-k5rr"
server_key: "44506fa6ae790cc348fc6de000640395"
server_key_iv: "4db0c53222d0d65071064ccc521ccfd3"
server_key_version: "encryptionkey@0"
I20260808 02:49:08.228276 25883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "824b425bc306441289621c4e13ee8243"
format_stamp: "Formatted at 2026-08-08 02:49:08 on dist-test-slave-k5rr"
server_key: "44506fa6ae790cc348fc6de000640395"
server_key_iv: "4db0c53222d0d65071064ccc521ccfd3"
server_key_version: "encryptionkey@0"
I20260808 02:49:08.233624 25883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260808 02:49:08.237284 25908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:08.238677 25883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:49:08.238854 25883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "824b425bc306441289621c4e13ee8243"
format_stamp: "Formatted at 2026-08-08 02:49:08 on dist-test-slave-k5rr"
server_key: "44506fa6ae790cc348fc6de000640395"
server_key_iv: "4db0c53222d0d65071064ccc521ccfd3"
server_key_version: "encryptionkey@0"
I20260808 02:49:08.239051 25883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:08.260466 25883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:08.264858 25883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:08.265168 25883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:08.282646 25883 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45999
I20260808 02:49:08.282701 26002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45999 every 8 connection(s)
I20260808 02:49:08.283998 25883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:08.287748 26004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:08.293044 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 25883
I20260808 02:49:08.293249 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:08.293881 26151 external_mini_cluster.cc:1428] Setting key 6e7a458c845326e962d647ca2a4e29bf
I20260808 02:49:08.298369 26004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243: Bootstrap starting.
I20260808 02:49:08.301908 26004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:08.303154 26004 log.cc:824] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:08.306290 26004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243: No bootstrap required, opened a new log
Aug 08 02:49:08 dist-test-slave-k5rr krb5kdc[25856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:49:08.315099 26004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824b425bc306441289621c4e13ee8243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45999 } }
I20260808 02:49:08.315474 26004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:08.315569 26004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824b425bc306441289621c4e13ee8243, State: Initialized, Role: FOLLOWER
I20260808 02:49:08.316306 26004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824b425bc306441289621c4e13ee8243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45999 } }
I20260808 02:49:08.316644 26004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:08.316803 26004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:08.316988 26004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:08.317832 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:08.295881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32829 (local address 127.0.0.1:45999)
0808 02:49:08.296682 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:08.296693 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:08.296732 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:49:08.298411 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:08.298434 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:08.298830 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:08.299127 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:08.300976 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:08.302440 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:08.304214 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:08.304952 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:08.305143 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:08.310204 (+ 5061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:08.310211 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:08.310225 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:08.310278 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:08.313371 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:08.314318 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:08.314324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:08.314330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:08.314486 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:08.314958 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:08.314961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:08.314962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:08.315490 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:08.315782 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:08.316253 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 02:49:08.316557 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":374,"threads_started":1}
I20260808 02:49:08.319530 26004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824b425bc306441289621c4e13ee8243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45999 } }
I20260808 02:49:08.319995 26004 leader_election.cc:302] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 824b425bc306441289621c4e13ee8243; no voters:
I20260808 02:49:08.320623 26004 leader_election.cc:288] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:08.320719 26009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:08.322008 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:08.322705 26004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:08.329859 26009 raft_consensus.cc:686] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [term 1 LEADER]: Becoming Leader. State: Replica: 824b425bc306441289621c4e13ee8243, State: Running, Role: LEADER
I20260808 02:49:08.330665 26009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824b425bc306441289621c4e13ee8243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45999 } }
I20260808 02:49:08.336890 26012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 824b425bc306441289621c4e13ee8243. Latest consensus state: current_term: 1 leader_uuid: "824b425bc306441289621c4e13ee8243" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824b425bc306441289621c4e13ee8243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45999 } } }
I20260808 02:49:08.337394 26012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:08.337946 26010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "824b425bc306441289621c4e13ee8243" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824b425bc306441289621c4e13ee8243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45999 } } }
I20260808 02:49:08.338208 26010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243 [sys.catalog]: This master's current role is: LEADER
W20260808 02:49:08.353542 26039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:49:08.353725 26039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:49:08.353881 26040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:08.355826 26040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:08.361310 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:08.333151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36143 (local address 127.0.0.1:45999)
0808 02:49:08.333347 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:08.333353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:08.335163 (+ 1810us) server_negotiation.cc:407] Connection header received
0808 02:49:08.336552 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:08.336558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:08.336640 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:08.336716 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:08.338834 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:08.340087 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:08.342153 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:08.347621 (+ 5468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:08.347727 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:08.351088 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:08.351095 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:08.351100 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:08.351148 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:08.357728 (+ 6580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:08.358809 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:08.358814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:08.358817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:08.358972 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:08.359578 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:08.359582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:08.359590 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:08.360017 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:08.360301 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:08.360736 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 02:49:08.360918 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:49:08.365645 26040 catalog_manager.cc:1347] Generated new cluster ID: 2fa3eb3f567249719089b57ca39be2bf
I20260808 02:49:08.365751 26040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:08.381913 26040 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:08.383729 26040 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:08.396736 26040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 824b425bc306441289621c4e13ee8243: Generated new TSK 0
I20260808 02:49:08.397890 26040 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:08.408485 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 25883
I20260808 02:49:08.409852 25997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:08.502468 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 25883
W20260808 02:49:08.526055 25853 connection.cc:535] client connection to 127.0.0.1:45999 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45999): Connection reset by peer (error 104)
2026-08-08T02:49:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2226 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:08 dist-test-slave-k5rr krb5kdc[26052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:08 dist-test-slave-k5rr krb5kdc[26052](info): set up 2 sockets
Aug 08 02:49:08 dist-test-slave-k5rr krb5kdc[26052](info): commencing operation
krb5kdc: starting...
W20260808 02:49:09.741569 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.007s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:09 dist-test-slave-k5rr krb5kdc[26052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:10.063585 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42401
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:10.399652 26073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:10.399964 26073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:10.400125 26073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:10.407471 26073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:10.407608 26073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:10.407685 26073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:10.407778 26073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:10.407838 26073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:10.417944 26073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42401
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:10.419344 26073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:10.420666 26073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:10.433375 26083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:10.433332 26089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:10.433339 26091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:10.438624 26073 server_base.cc:1034] running on GCE node
I20260808 02:49:10.439472 26073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:10.440959 26073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:10.442184 26073 hybrid_clock.cc:648] HybridClock initialized: now 1786157350442139 us; error 32 us; skew 500 ppm
I20260808 02:49:10.442530 26073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:10 dist-test-slave-k5rr krb5kdc[26052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157350, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:10.445757 26073 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:10.447232 26073 webserver.cc:460] Webserver started at http://127.0.0.1:45519/ using document root <none> and password file <none>
I20260808 02:49:10.447921 26073 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:10.448041 26073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:10.448411 26073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:10.451538 26073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "e4b1461bd11d497ea543416895ba1985"
format_stamp: "Formatted at 2026-08-08 02:49:10 on dist-test-slave-k5rr"
server_key: "97a9e108dc0ec4343ccdcca297881fb7"
server_key_iv: "54afd33c4ed8f598b38cc07595f1ea2a"
server_key_version: "encryptionkey@0"
I20260808 02:49:10.452332 26073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "e4b1461bd11d497ea543416895ba1985"
format_stamp: "Formatted at 2026-08-08 02:49:10 on dist-test-slave-k5rr"
server_key: "97a9e108dc0ec4343ccdcca297881fb7"
server_key_iv: "54afd33c4ed8f598b38cc07595f1ea2a"
server_key_version: "encryptionkey@0"
I20260808 02:49:10.457662 26073 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 02:49:10.461058 26100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:10.462395 26073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:49:10.462563 26073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "e4b1461bd11d497ea543416895ba1985"
format_stamp: "Formatted at 2026-08-08 02:49:10 on dist-test-slave-k5rr"
server_key: "97a9e108dc0ec4343ccdcca297881fb7"
server_key_iv: "54afd33c4ed8f598b38cc07595f1ea2a"
server_key_version: "encryptionkey@0"
I20260808 02:49:10.462743 26073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:10.479182 26073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:10.482995 26073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:10.483253 26073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:10.500627 26073 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42401
I20260808 02:49:10.500659 26188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42401 every 8 connection(s)
I20260808 02:49:10.501995 26073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:10.504494 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 26073
I20260808 02:49:10.504668 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:10.505128 26151 external_mini_cluster.cc:1428] Setting key bd83cb22f624ee1e16e7e688bda2359d
I20260808 02:49:10.506104 26190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:10.518335 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:10.506911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44543 (local address 127.0.0.1:42401)
0808 02:49:10.507681 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:10.507693 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:10.507733 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:49:10.509370 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:10.509393 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:10.509771 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:10.510079 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:10.510725 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:10.512114 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:10.513970 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:10.514618 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:10.514745 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:10.516032 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:10.516040 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:10.516052 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:10.516607 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:10.516687 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:10.516715 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:10.516853 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:10.517146 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 02:49:10.517329 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":349,"threads_started":1}
I20260808 02:49:10.519438 26190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985: Bootstrap starting.
I20260808 02:49:10.521989 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:10.522650 26190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:10.523844 26190 log.cc:824] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:10.526876 26190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985: No bootstrap required, opened a new log
I20260808 02:49:10.534123 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:10.527144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32835 (local address 127.0.0.1:42401)
0808 02:49:10.527341 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:10.527350 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:10.527762 (+ 412us) server_negotiation.cc:407] Connection header received
0808 02:49:10.527916 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:10.527921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:10.527995 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:10.528107 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:10.528676 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:10.529910 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:10.531355 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:10.532067 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:10.532187 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:10.533281 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:10.533287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:10.533302 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:10.533324 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:10.533368 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:10.533387 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:10.533494 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:10.533729 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 02:49:10.533785 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:49:10.536166 26190 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b1461bd11d497ea543416895ba1985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42401 } }
I20260808 02:49:10.536506 26190 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:10.536614 26190 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4b1461bd11d497ea543416895ba1985, State: Initialized, Role: FOLLOWER
I20260808 02:49:10.537215 26190 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b1461bd11d497ea543416895ba1985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42401 } }
I20260808 02:49:10.537438 26190 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:10.537578 26190 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:10.537755 26190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:10.540324 26190 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b1461bd11d497ea543416895ba1985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42401 } }
I20260808 02:49:10.540802 26190 leader_election.cc:302] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4b1461bd11d497ea543416895ba1985; no voters:
I20260808 02:49:10.541405 26190 leader_election.cc:288] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:10.541530 26208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:10.542200 26208 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [term 1 LEADER]: Becoming Leader. State: Replica: e4b1461bd11d497ea543416895ba1985, State: Running, Role: LEADER
I20260808 02:49:10.542740 26208 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b1461bd11d497ea543416895ba1985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42401 } }
I20260808 02:49:10.543313 26190 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:10.545750 26213 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4b1461bd11d497ea543416895ba1985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b1461bd11d497ea543416895ba1985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42401 } } }
I20260808 02:49:10.546141 26213 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:10.546660 26214 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4b1461bd11d497ea543416895ba1985. Latest consensus state: current_term: 1 leader_uuid: "e4b1461bd11d497ea543416895ba1985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b1461bd11d497ea543416895ba1985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42401 } } }
I20260808 02:49:10.547009 26214 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:10.549661 26222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:10.555850 26222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:10.564175 26222 catalog_manager.cc:1347] Generated new cluster ID: 647e2a7eccb94a638d6274730f81379c
I20260808 02:49:10.564278 26222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:10.610581 26222 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:10.611927 26222 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:10.618574 26222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4b1461bd11d497ea543416895ba1985: Generated new TSK 0
I20260808 02:49:10.619392 26222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:10.634541 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26073
W20260808 02:49:10.635610 26184 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44543
W20260808 02:49:10.636513 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26073: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:10.636678 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 26073
W20260808 02:49:10.660621 26049 connection.cc:535] client connection to 127.0.0.1:42401 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42401): Connection reset by peer (error 104)
2026-08-08T02:49:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2134 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:10 dist-test-slave-k5rr krb5kdc[26238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:10 dist-test-slave-k5rr krb5kdc[26238](info): set up 2 sockets
Aug 08 02:49:10 dist-test-slave-k5rr krb5kdc[26238](info): commencing operation
krb5kdc: starting...
W20260808 02:49:11.882192 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:12 dist-test-slave-k5rr krb5kdc[26238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:12.205343 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:12.541797 26260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:12.542101 26260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:12.542244 26260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:12.549439 26260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:12.549576 26260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:12.549657 26260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:12.549739 26260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:12.549813 26260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:12.559686 26260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41647
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:12.560998 26260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:12.562290 26260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:12.573484 26268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:12.573446 26272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:12.573995 26267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:12.574321 26260 server_base.cc:1034] running on GCE node
I20260808 02:49:12.575088 26260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:12.576424 26260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:12.577644 26260 hybrid_clock.cc:648] HybridClock initialized: now 1786157352577564 us; error 64 us; skew 500 ppm
I20260808 02:49:12.578022 26260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:12 dist-test-slave-k5rr krb5kdc[26238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157352, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:12.581199 26260 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:12.582665 26260 webserver.cc:460] Webserver started at http://127.0.0.1:39381/ using document root <none> and password file <none>
I20260808 02:49:12.583293 26260 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:12.583402 26260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:12.583657 26260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:12.586637 26260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "9f759ef28ea24cdd991da810b65ab4f8"
format_stamp: "Formatted at 2026-08-08 02:49:12 on dist-test-slave-k5rr"
server_key: "f0f83b79b4d7ea125defe80abee9302b"
server_key_iv: "90d105c17ef130a32ff9ec741a27ab06"
server_key_version: "encryptionkey@0"
I20260808 02:49:12.587361 26260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "9f759ef28ea24cdd991da810b65ab4f8"
format_stamp: "Formatted at 2026-08-08 02:49:12 on dist-test-slave-k5rr"
server_key: "f0f83b79b4d7ea125defe80abee9302b"
server_key_iv: "90d105c17ef130a32ff9ec741a27ab06"
server_key_version: "encryptionkey@0"
I20260808 02:49:12.592386 26260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:49:12.595904 26284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:12.597227 26260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:49:12.597397 26260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "9f759ef28ea24cdd991da810b65ab4f8"
format_stamp: "Formatted at 2026-08-08 02:49:12 on dist-test-slave-k5rr"
server_key: "f0f83b79b4d7ea125defe80abee9302b"
server_key_iv: "90d105c17ef130a32ff9ec741a27ab06"
server_key_version: "encryptionkey@0"
I20260808 02:49:12.597589 26260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:12.603425 26260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:12.607481 26260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:12.607784 26260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:12.626919 26260 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41647
I20260808 02:49:12.626948 26368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41647 every 8 connection(s)
I20260808 02:49:12.628403 26260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:12.632177 26369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:12.635569 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 26260
I20260808 02:49:12.635716 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:12.636137 26151 external_mini_cluster.cc:1428] Setting key dad211539efdc03877c5c22094c31a01
I20260808 02:49:12.644779 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:12.637943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56427 (local address 127.0.0.1:41647)
0808 02:49:12.638726 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:12.638742 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:12.638795 (+ 53us) server_negotiation.cc:407] Connection header received
0808 02:49:12.640850 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:12.640879 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:12.641369 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:12.641934 (+ 565us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:12.642923 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:12.642931 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:12.642947 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:12.642965 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:12.643038 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:12.643066 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:12.643306 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:12.643429 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 02:49:12.643603 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":301,"threads_started":1}
I20260808 02:49:12.645450 26369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8: Bootstrap starting.
I20260808 02:49:12.648439 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:12.648589 26369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:12.649860 26369 log.cc:824] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:12.653167 26369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8: No bootstrap required, opened a new log
I20260808 02:49:12.655937 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:12.653537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60199 (local address 127.0.0.1:41647)
0808 02:49:12.653719 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:12.653725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:12.654070 (+ 345us) server_negotiation.cc:407] Connection header received
0808 02:49:12.654199 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:12.654204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:12.654270 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:12.654386 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:12.655262 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:12.655267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:12.655271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:12.655290 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:12.655337 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:12.655361 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:12.655480 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:12.655600 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 02:49:12.655655 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:49:12.661623 26369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f759ef28ea24cdd991da810b65ab4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41647 } }
I20260808 02:49:12.662002 26369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:12.662096 26369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f759ef28ea24cdd991da810b65ab4f8, State: Initialized, Role: FOLLOWER
I20260808 02:49:12.662678 26369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f759ef28ea24cdd991da810b65ab4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41647 } }
I20260808 02:49:12.662961 26369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:12.663149 26369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:12.663383 26369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:12.666601 26369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f759ef28ea24cdd991da810b65ab4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41647 } }
I20260808 02:49:12.667235 26369 leader_election.cc:302] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f759ef28ea24cdd991da810b65ab4f8; no voters:
I20260808 02:49:12.667925 26369 leader_election.cc:288] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:12.668040 26387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:12.670442 26369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:12.672022 26387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [term 1 LEADER]: Becoming Leader. State: Replica: 9f759ef28ea24cdd991da810b65ab4f8, State: Running, Role: LEADER
I20260808 02:49:12.672636 26387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f759ef28ea24cdd991da810b65ab4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41647 } }
I20260808 02:49:12.676465 26389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f759ef28ea24cdd991da810b65ab4f8. Latest consensus state: current_term: 1 leader_uuid: "9f759ef28ea24cdd991da810b65ab4f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f759ef28ea24cdd991da810b65ab4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41647 } } }
I20260808 02:49:12.676891 26389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:12.677120 26388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f759ef28ea24cdd991da810b65ab4f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f759ef28ea24cdd991da810b65ab4f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41647 } } }
I20260808 02:49:12.677359 26388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:12.682238 26399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:12.687521 26399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:12.696722 26399 catalog_manager.cc:1347] Generated new cluster ID: 15abfcb4f2d04cf7b25d8de2296a1e04
I20260808 02:49:12.696866 26399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:12.712858 26399 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:12.713362 26399 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:12.721665 26399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f759ef28ea24cdd991da810b65ab4f8: Generated new TSK 0
I20260808 02:49:12.722647 26399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:12.739055 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26260
W20260808 02:49:12.740345 26363 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56427
W20260808 02:49:12.741000 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26260: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:12.741175 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 26260
2026-08-08T02:49:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2103 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:12 dist-test-slave-k5rr krb5kdc[26422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:12 dist-test-slave-k5rr krb5kdc[26422](info): set up 2 sockets
Aug 08 02:49:12 dist-test-slave-k5rr krb5kdc[26422](info): commencing operation
krb5kdc: starting...
W20260808 02:49:14.005084 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:14 dist-test-slave-k5rr krb5kdc[26422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:14.328330 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:14.663910 26446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:14.664187 26446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:14.664326 26446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:14.671391 26446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:14.671509 26446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:14.671581 26446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:14.671662 26446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:14.671722 26446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:14.681479 26446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:14.682723 26446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:14.684087 26446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:14.695852 26457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:14.695870 26458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:14.696125 26464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:14.696548 26446 server_base.cc:1034] running on GCE node
I20260808 02:49:14.697412 26446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:14.698928 26446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:14.700162 26446 hybrid_clock.cc:648] HybridClock initialized: now 1786157354700108 us; error 38 us; skew 500 ppm
I20260808 02:49:14.700529 26446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:14 dist-test-slave-k5rr krb5kdc[26422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157354, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:14.706755 26446 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:14.708370 26446 webserver.cc:460] Webserver started at http://127.0.0.1:33339/ using document root <none> and password file <none>
I20260808 02:49:14.709097 26446 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:14.709215 26446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:14.709483 26446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:14.712546 26446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "fc79962c49524ec3b6791579b6df2250"
format_stamp: "Formatted at 2026-08-08 02:49:14 on dist-test-slave-k5rr"
server_key: "26c8ab434a1e4311b3bb8e9e19b71db1"
server_key_iv: "59c590058e82a7e150dc72a28813feee"
server_key_version: "encryptionkey@0"
I20260808 02:49:14.713337 26446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "fc79962c49524ec3b6791579b6df2250"
format_stamp: "Formatted at 2026-08-08 02:49:14 on dist-test-slave-k5rr"
server_key: "26c8ab434a1e4311b3bb8e9e19b71db1"
server_key_iv: "59c590058e82a7e150dc72a28813feee"
server_key_version: "encryptionkey@0"
I20260808 02:49:14.718570 26446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 02:49:14.722054 26473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:14.723352 26446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:49:14.723508 26446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "fc79962c49524ec3b6791579b6df2250"
format_stamp: "Formatted at 2026-08-08 02:49:14 on dist-test-slave-k5rr"
server_key: "26c8ab434a1e4311b3bb8e9e19b71db1"
server_key_iv: "59c590058e82a7e150dc72a28813feee"
server_key_version: "encryptionkey@0"
I20260808 02:49:14.723683 26446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:14.752477 26446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:14.756465 26446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:14.756740 26446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:14.774840 26446 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38853
I20260808 02:49:14.774896 26548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38853 every 8 connection(s)
I20260808 02:49:14.776203 26446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:14.779186 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 26446
I20260808 02:49:14.779350 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:14.779762 26151 external_mini_cluster.cc:1428] Setting key 0ce281696034693b9991a4b4339d379b
I20260808 02:49:14.780063 26551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:14.794595 26551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250: Bootstrap starting.
Aug 08 02:49:14 dist-test-slave-k5rr krb5kdc[26422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:49:14.797848 26551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:14.798934 26551 log.cc:824] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:14.801723 26551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250: No bootstrap required, opened a new log
I20260808 02:49:14.803189 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:14.781510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36505 (local address 127.0.0.1:38853)
0808 02:49:14.782373 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:14.782384 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:14.782426 (+ 42us) server_negotiation.cc:407] Connection header received
0808 02:49:14.784682 (+ 2256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:14.784703 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:14.785122 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:14.785479 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:14.787241 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:14.788523 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:14.790312 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:14.791026 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:14.791176 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:14.796149 (+ 4973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:14.796158 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:14.796173 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:14.796225 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:14.799264 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:14.799948 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:14.799955 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:14.799962 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:14.800145 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:14.800524 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:14.800528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:14.800530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:14.801097 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:14.801409 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:14.801832 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 02:49:14.802230 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":340,"threads_started":1}
I20260808 02:49:14.806701 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:14.809508 26551 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc79962c49524ec3b6791579b6df2250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38853 } }
I20260808 02:49:14.809938 26551 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:14.810037 26551 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc79962c49524ec3b6791579b6df2250, State: Initialized, Role: FOLLOWER
I20260808 02:49:14.810580 26551 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc79962c49524ec3b6791579b6df2250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38853 } }
I20260808 02:49:14.810807 26551 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:14.810993 26551 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:14.811164 26551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:14.814216 26551 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc79962c49524ec3b6791579b6df2250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38853 } }
I20260808 02:49:14.814716 26551 leader_election.cc:302] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc79962c49524ec3b6791579b6df2250; no voters:
I20260808 02:49:14.815376 26551 leader_election.cc:288] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:14.815454 26563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:14.817276 26563 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [term 1 LEADER]: Becoming Leader. State: Replica: fc79962c49524ec3b6791579b6df2250, State: Running, Role: LEADER
I20260808 02:49:14.817595 26551 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:14.817862 26563 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc79962c49524ec3b6791579b6df2250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38853 } }
I20260808 02:49:14.827248 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:14.810967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43545 (local address 127.0.0.1:38853)
0808 02:49:14.811151 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:14.811161 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:14.811706 (+ 545us) server_negotiation.cc:407] Connection header received
0808 02:49:14.811884 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:14.811887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:14.811945 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:14.812027 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:14.813766 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:14.814545 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:14.816259 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:14.816929 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:14.817054 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:14.820263 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:14.820270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:14.820276 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:14.820316 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:14.824214 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:14.825034 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:14.825047 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:14.825052 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:14.825227 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:14.825647 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:14.825651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:14.825654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:14.826082 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:14.826351 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:14.826676 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 02:49:14.826895 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:49:14.829279 26568 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc79962c49524ec3b6791579b6df2250. Latest consensus state: current_term: 1 leader_uuid: "fc79962c49524ec3b6791579b6df2250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc79962c49524ec3b6791579b6df2250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38853 } } }
I20260808 02:49:14.829293 26566 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc79962c49524ec3b6791579b6df2250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc79962c49524ec3b6791579b6df2250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38853 } } }
I20260808 02:49:14.829885 26568 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:14.829885 26566 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:14.834697 26579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:14.839352 26579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:14.847355 26579 catalog_manager.cc:1347] Generated new cluster ID: 61697c0de8e143ba9e69efc8347eccd2
I20260808 02:49:14.847476 26579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:14.857573 26579 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:14.859367 26579 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:14.870018 26579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc79962c49524ec3b6791579b6df2250: Generated new TSK 0
I20260808 02:49:14.871032 26579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:14.885952 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26446
I20260808 02:49:14.887061 26539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:14.978003 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 26446
2026-08-08T02:49:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2233 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:15 dist-test-slave-k5rr krb5kdc[26594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:15 dist-test-slave-k5rr krb5kdc[26594](info): set up 2 sockets
Aug 08 02:49:15 dist-test-slave-k5rr krb5kdc[26594](info): commencing operation
krb5kdc: starting...
W20260808 02:49:16.214728 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:16 dist-test-slave-k5rr krb5kdc[26594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:16.538085 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:16.873342 26613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:16.873631 26613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:16.873764 26613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:16.880983 26613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:16.881115 26613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:16.881192 26613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:16.881265 26613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:16.881320 26613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:16.891126 26613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:16.892409 26613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:16.893775 26613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:16.905045 26626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:16.905086 26630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:16.905115 26628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:16.906104 26613 server_base.cc:1034] running on GCE node
I20260808 02:49:16.906860 26613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:16.908160 26613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:16.909353 26613 hybrid_clock.cc:648] HybridClock initialized: now 1786157356909310 us; error 40 us; skew 500 ppm
I20260808 02:49:16.909662 26613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:16 dist-test-slave-k5rr krb5kdc[26594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157356, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:16.912693 26613 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:16.914186 26613 webserver.cc:460] Webserver started at http://127.0.0.1:41769/ using document root <none> and password file <none>
I20260808 02:49:16.914808 26613 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:16.914924 26613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:16.915201 26613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:16.918169 26613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "dbf86d81618b417d8abfde5fa9ac6b52"
format_stamp: "Formatted at 2026-08-08 02:49:16 on dist-test-slave-k5rr"
server_key: "51ec72ad00e091f87c25e76879b0fddf"
server_key_iv: "a32524572d63b75f0cece8cc8e600c3c"
server_key_version: "encryptionkey@0"
I20260808 02:49:16.918919 26613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "dbf86d81618b417d8abfde5fa9ac6b52"
format_stamp: "Formatted at 2026-08-08 02:49:16 on dist-test-slave-k5rr"
server_key: "51ec72ad00e091f87c25e76879b0fddf"
server_key_iv: "a32524572d63b75f0cece8cc8e600c3c"
server_key_version: "encryptionkey@0"
I20260808 02:49:16.923836 26613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:49:16.927174 26643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:16.928457 26613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 02:49:16.928606 26613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "dbf86d81618b417d8abfde5fa9ac6b52"
format_stamp: "Formatted at 2026-08-08 02:49:16 on dist-test-slave-k5rr"
server_key: "51ec72ad00e091f87c25e76879b0fddf"
server_key_iv: "a32524572d63b75f0cece8cc8e600c3c"
server_key_version: "encryptionkey@0"
I20260808 02:49:16.928769 26613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:16.940546 26613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:16.944499 26613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:16.944787 26613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:16.963845 26613 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43067
I20260808 02:49:16.963881 26727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43067 every 8 connection(s)
I20260808 02:49:16.965378 26613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:16.969105 26728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:16.970309 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 26613
I20260808 02:49:16.970501 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:16.971050 26151 external_mini_cluster.cc:1428] Setting key 7bc658872acabbd2560fcd42539ad7f5
I20260808 02:49:16.982420 26728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52: Bootstrap starting.
I20260808 02:49:16.984020 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:16.972843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54747 (local address 127.0.0.1:43067)
0808 02:49:16.973657 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:16.973668 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:16.973711 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:49:16.975355 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:16.975376 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:16.975753 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:16.976084 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:16.976666 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:16.977986 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:16.979765 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:16.980482 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:16.980606 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:16.981828 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:16.981835 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:16.981848 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:16.982370 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:16.982451 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:16.982477 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:16.982588 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:16.982789 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 02:49:16.983049 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":343,"threads_started":1}
I20260808 02:49:16.985610 26728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:16.986685 26728 log.cc:824] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:16.987516 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:16.989711 26728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52: No bootstrap required, opened a new log
I20260808 02:49:16.999349 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:16.992665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43491 (local address 127.0.0.1:43067)
0808 02:49:16.992864 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:16.992870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:16.993199 (+ 329us) server_negotiation.cc:407] Connection header received
0808 02:49:16.993387 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:16.993391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:16.993450 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:16.993532 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:16.994196 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:16.995030 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:16.996855 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:16.997392 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:16.997501 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:16.998640 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:16.998645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:16.998655 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:16.998674 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:16.998715 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:16.998732 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:16.998817 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:16.998950 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 02:49:16.999025 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 02:49:16.999536 26728 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf86d81618b417d8abfde5fa9ac6b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260808 02:49:16.999940 26728 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:17.000065 26728 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf86d81618b417d8abfde5fa9ac6b52, State: Initialized, Role: FOLLOWER
I20260808 02:49:17.000627 26728 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf86d81618b417d8abfde5fa9ac6b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260808 02:49:17.000865 26728 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:17.000990 26728 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:17.001185 26728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:17.003306 26728 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf86d81618b417d8abfde5fa9ac6b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260808 02:49:17.003811 26728 leader_election.cc:302] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbf86d81618b417d8abfde5fa9ac6b52; no voters:
I20260808 02:49:17.004424 26728 leader_election.cc:288] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:17.004534 26747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:17.005319 26747 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [term 1 LEADER]: Becoming Leader. State: Replica: dbf86d81618b417d8abfde5fa9ac6b52, State: Running, Role: LEADER
I20260808 02:49:17.005995 26747 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf86d81618b417d8abfde5fa9ac6b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260808 02:49:17.006949 26728 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:17.009114 26749 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbf86d81618b417d8abfde5fa9ac6b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf86d81618b417d8abfde5fa9ac6b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } } }
I20260808 02:49:17.009178 26750 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbf86d81618b417d8abfde5fa9ac6b52. Latest consensus state: current_term: 1 leader_uuid: "dbf86d81618b417d8abfde5fa9ac6b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf86d81618b417d8abfde5fa9ac6b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } } }
I20260808 02:49:17.009641 26750 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:17.009639 26749 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:17.017853 26756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:17.022154 26756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:17.031076 26756 catalog_manager.cc:1347] Generated new cluster ID: 122c02a4e9864835920b5a52c8c0b506
I20260808 02:49:17.031201 26756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:17.049540 26756 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:17.051450 26756 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:17.073139 26756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbf86d81618b417d8abfde5fa9ac6b52: Generated new TSK 0
I20260808 02:49:17.074038 26756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:17.088029 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26613
W20260808 02:49:17.089530 26722 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54747
W20260808 02:49:17.090392 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26613: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:17.090554 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 26613
2026-08-08T02:49:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2118 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:17 dist-test-slave-k5rr krb5kdc[26780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:17 dist-test-slave-k5rr krb5kdc[26780](info): set up 2 sockets
Aug 08 02:49:17 dist-test-slave-k5rr krb5kdc[26780](info): commencing operation
krb5kdc: starting...
W20260808 02:49:18.336534 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:18 dist-test-slave-k5rr krb5kdc[26780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:18.668164 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37093
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:19.001597 26800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:19.001914 26800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:19.002063 26800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:19.009338 26800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:19.009478 26800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:19.009557 26800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:19.009641 26800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:19.009701 26800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:19.019749 26800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:19.021121 26800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:19.022483 26800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:19.034001 26805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:19.033957 26809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:19.033952 26806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:19.035267 26800 server_base.cc:1034] running on GCE node
I20260808 02:49:19.036005 26800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:19.037431 26800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:19.038640 26800 hybrid_clock.cc:648] HybridClock initialized: now 1786157359038595 us; error 43 us; skew 500 ppm
I20260808 02:49:19.039016 26800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:19 dist-test-slave-k5rr krb5kdc[26780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157359, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:19.042061 26800 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:19.043519 26800 webserver.cc:460] Webserver started at http://127.0.0.1:42629/ using document root <none> and password file <none>
I20260808 02:49:19.044152 26800 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:19.044269 26800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:19.044553 26800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:19.047525 26800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "e383366998e24e2b98fb26c7e97af179"
format_stamp: "Formatted at 2026-08-08 02:49:19 on dist-test-slave-k5rr"
server_key: "c5bc9c5c742e186969ddc7b177054453"
server_key_iv: "d637eb992ba5023041e7d4f4d90f069a"
server_key_version: "encryptionkey@0"
I20260808 02:49:19.048285 26800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "e383366998e24e2b98fb26c7e97af179"
format_stamp: "Formatted at 2026-08-08 02:49:19 on dist-test-slave-k5rr"
server_key: "c5bc9c5c742e186969ddc7b177054453"
server_key_iv: "d637eb992ba5023041e7d4f4d90f069a"
server_key_version: "encryptionkey@0"
I20260808 02:49:19.053332 26800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:49:19.056699 26821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:19.058063 26800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:49:19.058225 26800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "e383366998e24e2b98fb26c7e97af179"
format_stamp: "Formatted at 2026-08-08 02:49:19 on dist-test-slave-k5rr"
server_key: "c5bc9c5c742e186969ddc7b177054453"
server_key_iv: "d637eb992ba5023041e7d4f4d90f069a"
server_key_version: "encryptionkey@0"
I20260808 02:49:19.058413 26800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:19.064117 26800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:19.068127 26800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:19.068410 26800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:19.086424 26800 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37093
I20260808 02:49:19.086493 26903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37093 every 8 connection(s)
I20260808 02:49:19.087800 26800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:19.089805 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 26800
I20260808 02:49:19.089990 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:19.090570 26151 external_mini_cluster.cc:1428] Setting key ef96b6765e04324343f7ed9b5d2f6e79
I20260808 02:49:19.092327 26904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:19.098234 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:19.092429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45193 (local address 127.0.0.1:37093)
0808 02:49:19.093245 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:19.093264 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:19.093302 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:49:19.094889 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:19.094911 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:19.095243 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:19.095570 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:19.096644 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:19.096651 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:19.096663 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:19.096680 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:19.096764 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:19.096788 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:19.096981 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:19.097122 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 02:49:19.097332 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":348,"threads_started":1}
I20260808 02:49:19.101799 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:19.105278 26904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179: Bootstrap starting.
I20260808 02:49:19.109303 26904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:19.109807 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:19.107285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48035 (local address 127.0.0.1:37093)
0808 02:49:19.107456 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:19.107461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:19.107849 (+ 388us) server_negotiation.cc:407] Connection header received
0808 02:49:19.108016 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:19.108019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:19.108068 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:19.108147 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:19.109128 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:19.109133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:19.109136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:19.109150 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:19.109192 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:19.109208 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:19.109298 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:19.109460 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 02:49:19.109518 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 02:49:19.110555 26904 log.cc:824] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:19.113300 26904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179: No bootstrap required, opened a new log
I20260808 02:49:19.120425 26904 raft_consensus.cc:348] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e383366998e24e2b98fb26c7e97af179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } }
I20260808 02:49:19.120803 26904 raft_consensus.cc:374] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:19.120914 26904 raft_consensus.cc:729] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e383366998e24e2b98fb26c7e97af179, State: Initialized, Role: FOLLOWER
I20260808 02:49:19.121544 26904 consensus_queue.cc:260] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e383366998e24e2b98fb26c7e97af179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } }
I20260808 02:49:19.121798 26904 raft_consensus.cc:388] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:19.121924 26904 raft_consensus.cc:482] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:19.122104 26904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:19.124352 26904 raft_consensus.cc:504] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e383366998e24e2b98fb26c7e97af179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } }
I20260808 02:49:19.124859 26904 leader_election.cc:302] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e383366998e24e2b98fb26c7e97af179; no voters:
I20260808 02:49:19.125535 26904 leader_election.cc:288] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:19.125669 26918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:19.126407 26918 raft_consensus.cc:686] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [term 1 LEADER]: Becoming Leader. State: Replica: e383366998e24e2b98fb26c7e97af179, State: Running, Role: LEADER
I20260808 02:49:19.126942 26918 consensus_queue.cc:237] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e383366998e24e2b98fb26c7e97af179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } }
I20260808 02:49:19.127609 26904 sys_catalog.cc:564] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:19.129853 26919 sys_catalog.cc:455] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e383366998e24e2b98fb26c7e97af179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e383366998e24e2b98fb26c7e97af179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } } }
I20260808 02:49:19.132925 26920 sys_catalog.cc:455] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e383366998e24e2b98fb26c7e97af179. Latest consensus state: current_term: 1 leader_uuid: "e383366998e24e2b98fb26c7e97af179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e383366998e24e2b98fb26c7e97af179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37093 } } }
I20260808 02:49:19.133338 26919 sys_catalog.cc:458] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:19.133347 26920 sys_catalog.cc:458] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:19.135197 26930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:19.140663 26930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:19.149705 26930 catalog_manager.cc:1347] Generated new cluster ID: 3dd84a8bbc844dcaa25ca84d1b6328cb
I20260808 02:49:19.149839 26930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:19.179505 26930 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:19.179942 26930 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:19.200397 26930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e383366998e24e2b98fb26c7e97af179: Generated new TSK 0
I20260808 02:49:19.201272 26930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:19.220849 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26800
W20260808 02:49:19.222019 26896 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45193
W20260808 02:49:19.222811 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26800: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:19.222959 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 26800
2026-08-08T02:49:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2133 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:19 dist-test-slave-k5rr krb5kdc[26948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:19 dist-test-slave-k5rr krb5kdc[26948](info): set up 2 sockets
Aug 08 02:49:19 dist-test-slave-k5rr krb5kdc[26948](info): commencing operation
krb5kdc: starting...
W20260808 02:49:20.475378 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.005s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:20 dist-test-slave-k5rr krb5kdc[26948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:20.800612 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39545
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:21.138082 26967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:21.138370 26967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:21.138504 26967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:21.145650 26967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:21.145752 26967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:21.145833 26967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:21.145923 26967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:21.145982 26967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:21.155716 26967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:21.156936 26967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:21.158216 26967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:21.169354 26972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:21.169418 26975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:21.169384 26977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:21.170791 26967 server_base.cc:1034] running on GCE node
I20260808 02:49:21.171600 26967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:21.172994 26967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:21.174278 26967 hybrid_clock.cc:648] HybridClock initialized: now 1786157361174216 us; error 40 us; skew 500 ppm
I20260808 02:49:21.174639 26967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:21 dist-test-slave-k5rr krb5kdc[26948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157361, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:21.181810 26967 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:21.183364 26967 webserver.cc:460] Webserver started at http://127.0.0.1:42763/ using document root <none> and password file <none>
I20260808 02:49:21.184031 26967 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:21.184156 26967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:21.184450 26967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:21.187477 26967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "accbf881e3114ba5a60904155052e79e"
format_stamp: "Formatted at 2026-08-08 02:49:21 on dist-test-slave-k5rr"
server_key: "f96852e8ff2f47cca2d1227390ceb30c"
server_key_iv: "d5f9dcca7676ee2d607ba71bc0ac42c3"
server_key_version: "encryptionkey@0"
I20260808 02:49:21.188380 26967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "accbf881e3114ba5a60904155052e79e"
format_stamp: "Formatted at 2026-08-08 02:49:21 on dist-test-slave-k5rr"
server_key: "f96852e8ff2f47cca2d1227390ceb30c"
server_key_iv: "d5f9dcca7676ee2d607ba71bc0ac42c3"
server_key_version: "encryptionkey@0"
I20260808 02:49:21.193910 26967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 02:49:21.197378 26987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:21.198643 26967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 02:49:21.198828 26967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "accbf881e3114ba5a60904155052e79e"
format_stamp: "Formatted at 2026-08-08 02:49:21 on dist-test-slave-k5rr"
server_key: "f96852e8ff2f47cca2d1227390ceb30c"
server_key_iv: "d5f9dcca7676ee2d607ba71bc0ac42c3"
server_key_version: "encryptionkey@0"
I20260808 02:49:21.199025 26967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:21.212674 26967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:21.216928 26967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:21.217253 26967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:21.237071 26967 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39545
I20260808 02:49:21.237092 27079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39545 every 8 connection(s)
I20260808 02:49:21.238564 26967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:21.242362 27080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:21.242790 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 26967
I20260808 02:49:21.242980 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:21.243405 26151 external_mini_cluster.cc:1428] Setting key d34278c2d5056de688fb0859bae49926
I20260808 02:49:21.256356 27080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e: Bootstrap starting.
Aug 08 02:49:21 dist-test-slave-k5rr krb5kdc[26948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:49:21.260430 27080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:21.261658 27080 log.cc:824] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:21.265041 27080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e: No bootstrap required, opened a new log
I20260808 02:49:21.269222 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:21.245237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:42949 (local address 127.0.0.1:39545)
0808 02:49:21.245967 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:21.245978 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:21.246021 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:49:21.248206 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:21.248231 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:21.248639 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:21.248963 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:21.251040 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:21.252345 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:21.254292 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:21.255124 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:21.255247 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:21.260583 (+ 5336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:21.260593 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:21.260610 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:21.260648 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:21.264169 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:21.264978 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:21.264988 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:21.264996 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:21.265252 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:21.265745 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:21.265750 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:21.265754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:21.266487 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:21.266858 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:21.267349 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 02:49:21.267710 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":324,"threads_started":1}
I20260808 02:49:21.273087 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:21.275044 27080 raft_consensus.cc:348] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "accbf881e3114ba5a60904155052e79e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39545 } }
I20260808 02:49:21.275568 27080 raft_consensus.cc:374] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:21.275686 27080 raft_consensus.cc:729] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: accbf881e3114ba5a60904155052e79e, State: Initialized, Role: FOLLOWER
I20260808 02:49:21.276396 27080 consensus_queue.cc:260] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "accbf881e3114ba5a60904155052e79e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39545 } }
I20260808 02:49:21.276718 27080 raft_consensus.cc:388] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:21.276891 27080 raft_consensus.cc:482] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:21.277104 27080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:21.280061 27080 raft_consensus.cc:504] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "accbf881e3114ba5a60904155052e79e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39545 } }
I20260808 02:49:21.280561 27080 leader_election.cc:302] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: accbf881e3114ba5a60904155052e79e; no voters:
I20260808 02:49:21.281333 27080 leader_election.cc:288] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:21.281425 27094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:21.283334 27080 sys_catalog.cc:564] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:21.283738 27094 raft_consensus.cc:686] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [term 1 LEADER]: Becoming Leader. State: Replica: accbf881e3114ba5a60904155052e79e, State: Running, Role: LEADER
I20260808 02:49:21.284355 27094 consensus_queue.cc:237] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "accbf881e3114ba5a60904155052e79e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39545 } }
I20260808 02:49:21.288084 27096 sys_catalog.cc:455] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [sys.catalog]: SysCatalogTable state changed. Reason: New leader accbf881e3114ba5a60904155052e79e. Latest consensus state: current_term: 1 leader_uuid: "accbf881e3114ba5a60904155052e79e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "accbf881e3114ba5a60904155052e79e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39545 } } }
I20260808 02:49:21.288506 27096 sys_catalog.cc:458] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:21.288074 27095 sys_catalog.cc:455] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "accbf881e3114ba5a60904155052e79e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "accbf881e3114ba5a60904155052e79e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39545 } } }
I20260808 02:49:21.288908 27095 sys_catalog.cc:458] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:21.290535 27107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:21.295492 27107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:21.298465 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:21.278494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:40381 (local address 127.0.0.1:39545)
0808 02:49:21.278678 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:21.278684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:21.279185 (+ 501us) server_negotiation.cc:407] Connection header received
0808 02:49:21.279306 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:21.279312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:21.279393 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:21.279508 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:21.280974 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:21.282274 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:21.284178 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:21.284991 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:21.285187 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:21.289000 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:21.289031 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:21.289037 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:21.289077 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:21.293266 (+ 4189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:21.294461 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:21.294468 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:21.294473 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:21.294647 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:21.295332 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:21.295339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:21.295343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:21.295844 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:21.296133 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:21.297806 (+ 1673us) server_negotiation.cc:299] Negotiation successful
0808 02:49:21.298020 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:49:21.311367 27107 catalog_manager.cc:1347] Generated new cluster ID: cd3367554db94cba83863c7478d97492
I20260808 02:49:21.311546 27107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:21.324896 27107 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:21.326697 27107 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:21.339119 27107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P accbf881e3114ba5a60904155052e79e: Generated new TSK 0
I20260808 02:49:21.340174 27107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:21.350492 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 26967
I20260808 02:49:21.351684 27073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:21.442377 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 26967
W20260808 02:49:21.466518 26945 connection.cc:535] client connection to 127.0.0.1:39545 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39545): Connection reset by peer (error 104)
2026-08-08T02:49:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2216 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:21 dist-test-slave-k5rr krb5kdc[27130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:21 dist-test-slave-k5rr krb5kdc[27130](info): set up 2 sockets
Aug 08 02:49:21 dist-test-slave-k5rr krb5kdc[27130](info): commencing operation
krb5kdc: starting...
W20260808 02:49:22.729802 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:22 dist-test-slave-k5rr krb5kdc[27130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:23.049088 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:23.387158 27153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:23.387430 27153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:23.387573 27153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:23.394840 27153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:23.394953 27153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:23.395032 27153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:23.395123 27153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:23.395181 27153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:23.405267 27153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:23.406558 27153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:23.407893 27153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:23.419909 27161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:23.419809 27165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:23.419839 27163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:23.420322 27153 server_base.cc:1034] running on GCE node
I20260808 02:49:23.421330 27153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:23.422643 27153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:23.423846 27153 hybrid_clock.cc:648] HybridClock initialized: now 1786157363423795 us; error 47 us; skew 500 ppm
I20260808 02:49:23.424178 27153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:23 dist-test-slave-k5rr krb5kdc[27130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157363, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:23.427415 27153 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:23.428865 27153 webserver.cc:460] Webserver started at http://127.0.0.1:41759/ using document root <none> and password file <none>
I20260808 02:49:23.429540 27153 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:23.429661 27153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:23.429949 27153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:23.432937 27153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "34d877d5fe5d4b52a2c48836e9304516"
format_stamp: "Formatted at 2026-08-08 02:49:23 on dist-test-slave-k5rr"
server_key: "61587ccbc3f050f22d5b17bb52b39d89"
server_key_iv: "cebf72be624fb8f1c25644077555269f"
server_key_version: "encryptionkey@0"
I20260808 02:49:23.433765 27153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "34d877d5fe5d4b52a2c48836e9304516"
format_stamp: "Formatted at 2026-08-08 02:49:23 on dist-test-slave-k5rr"
server_key: "61587ccbc3f050f22d5b17bb52b39d89"
server_key_iv: "cebf72be624fb8f1c25644077555269f"
server_key_version: "encryptionkey@0"
I20260808 02:49:23.438707 27153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:49:23.442703 27176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:23.444082 27153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:49:23.444265 27153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "34d877d5fe5d4b52a2c48836e9304516"
format_stamp: "Formatted at 2026-08-08 02:49:23 on dist-test-slave-k5rr"
server_key: "61587ccbc3f050f22d5b17bb52b39d89"
server_key_iv: "cebf72be624fb8f1c25644077555269f"
server_key_version: "encryptionkey@0"
I20260808 02:49:23.444484 27153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:23.468977 27153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:23.473325 27153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:23.473601 27153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:23.494571 27153 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38981
I20260808 02:49:23.494596 27262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38981 every 8 connection(s)
I20260808 02:49:23.495918 27153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:23.499590 27263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:23.501279 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 27153
I20260808 02:49:23.501437 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:23.501863 26151 external_mini_cluster.cc:1428] Setting key 4b7256e1e9da7ad807713d917899b7a3
I20260808 02:49:23.512939 27263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516: Bootstrap starting.
I20260808 02:49:23.514806 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:23.503600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:52457 (local address 127.0.0.1:38981)
0808 02:49:23.504392 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:23.504403 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:23.504443 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:49:23.506076 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:23.506101 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:23.506495 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:23.506789 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:23.507365 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:23.508601 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:23.510526 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:23.511174 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:23.511317 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:23.512569 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:23.512577 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:23.512589 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:23.513160 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:23.513237 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:23.513262 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:23.513383 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:23.513662 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 02:49:23.513824 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":346,"threads_started":1}
I20260808 02:49:23.516280 27263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:23.517565 27263 log.cc:824] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:23.518674 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:23.520174 27263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516: No bootstrap required, opened a new log
I20260808 02:49:23.529292 27263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d877d5fe5d4b52a2c48836e9304516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260808 02:49:23.529718 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:23.522894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:52097 (local address 127.0.0.1:38981)
0808 02:49:23.523081 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:23.523086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:23.523388 (+ 302us) server_negotiation.cc:407] Connection header received
0808 02:49:23.523570 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:23.523573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:23.523634 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:23.523719 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:23.524332 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:23.525210 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:23.527055 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:23.527572 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:23.527686 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:23.528865 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:23.528870 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:23.528873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:23.528891 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:23.528949 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:23.528967 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:23.529127 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:23.529297 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 02:49:23.529346 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:49:23.529722 27263 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:23.529884 27263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34d877d5fe5d4b52a2c48836e9304516, State: Initialized, Role: FOLLOWER
I20260808 02:49:23.530472 27263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d877d5fe5d4b52a2c48836e9304516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260808 02:49:23.530758 27263 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:23.530917 27263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:23.531100 27263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:23.533265 27263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d877d5fe5d4b52a2c48836e9304516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260808 02:49:23.533726 27263 leader_election.cc:302] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34d877d5fe5d4b52a2c48836e9304516; no voters:
I20260808 02:49:23.534288 27263 leader_election.cc:288] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:23.534409 27282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:23.535126 27282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [term 1 LEADER]: Becoming Leader. State: Replica: 34d877d5fe5d4b52a2c48836e9304516, State: Running, Role: LEADER
I20260808 02:49:23.535676 27282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d877d5fe5d4b52a2c48836e9304516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260808 02:49:23.536201 27263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:23.538427 27285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34d877d5fe5d4b52a2c48836e9304516. Latest consensus state: current_term: 1 leader_uuid: "34d877d5fe5d4b52a2c48836e9304516" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d877d5fe5d4b52a2c48836e9304516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } } }
I20260808 02:49:23.538897 27285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:23.538718 27284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34d877d5fe5d4b52a2c48836e9304516" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d877d5fe5d4b52a2c48836e9304516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } } }
I20260808 02:49:23.539059 27284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:23.539810 27293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:23.545397 27293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:23.555384 27293 catalog_manager.cc:1347] Generated new cluster ID: d5146228ffb441af8aac9c38b5e104d7
I20260808 02:49:23.555505 27293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:23.562151 27293 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:23.563894 27293 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:23.588904 27293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34d877d5fe5d4b52a2c48836e9304516: Generated new TSK 0
I20260808 02:49:23.589830 27293 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:23.597865 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27153
W20260808 02:49:23.599501 27257 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.199:52457
W20260808 02:49:23.600317 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27153: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:23.600461 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 27153
W20260808 02:49:23.627175 27128 connection.cc:535] client connection to 127.0.0.1:38981 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38981): Connection reset by peer (error 104)
2026-08-08T02:49:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2160 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:23 dist-test-slave-k5rr krb5kdc[27308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:23 dist-test-slave-k5rr krb5kdc[27308](info): set up 2 sockets
Aug 08 02:49:23 dist-test-slave-k5rr krb5kdc[27308](info): commencing operation
krb5kdc: starting...
W20260808 02:49:24.846549 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.007s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:25 dist-test-slave-k5rr krb5kdc[27308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:25.179646 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:25.524669 27328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:25.525027 27328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:25.525202 27328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:25.532639 27328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:25.532765 27328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:25.532850 27328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:25.532948 27328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:25.533038 27328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:25.543000 27328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:25.544260 27328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:25.545670 27328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:25.557632 27339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:25.557586 27345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:25.557812 27340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:25.558867 27328 server_base.cc:1034] running on GCE node
I20260808 02:49:25.559620 27328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:25.561025 27328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:25.562274 27328 hybrid_clock.cc:648] HybridClock initialized: now 1786157365562228 us; error 36 us; skew 500 ppm
I20260808 02:49:25.562608 27328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:25 dist-test-slave-k5rr krb5kdc[27308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157365, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:25.565817 27328 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:25.567335 27328 webserver.cc:460] Webserver started at http://127.0.0.1:34513/ using document root <none> and password file <none>
I20260808 02:49:25.568004 27328 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:25.568122 27328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:25.568416 27328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:25.571441 27328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "019859189f434f8e87ebfc9945e61723"
format_stamp: "Formatted at 2026-08-08 02:49:25 on dist-test-slave-k5rr"
server_key: "12504f42ac1b03906d53bf0bd6a5f3ae"
server_key_iv: "53eb31b9b4ce502cf2500db63947416b"
server_key_version: "encryptionkey@0"
I20260808 02:49:25.572228 27328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "019859189f434f8e87ebfc9945e61723"
format_stamp: "Formatted at 2026-08-08 02:49:25 on dist-test-slave-k5rr"
server_key: "12504f42ac1b03906d53bf0bd6a5f3ae"
server_key_iv: "53eb31b9b4ce502cf2500db63947416b"
server_key_version: "encryptionkey@0"
I20260808 02:49:25.577600 27328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 02:49:25.581210 27361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:25.582530 27328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:49:25.582702 27328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "019859189f434f8e87ebfc9945e61723"
format_stamp: "Formatted at 2026-08-08 02:49:25 on dist-test-slave-k5rr"
server_key: "12504f42ac1b03906d53bf0bd6a5f3ae"
server_key_iv: "53eb31b9b4ce502cf2500db63947416b"
server_key_version: "encryptionkey@0"
I20260808 02:49:25.582894 27328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:25.589275 27328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:25.593358 27328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:25.593636 27328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:25.611462 27328 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38099
I20260808 02:49:25.611513 27440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38099 every 8 connection(s)
I20260808 02:49:25.612859 27328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:25.616591 27441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:25.622429 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 27328
I20260808 02:49:25.622620 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:25.623170 26151 external_mini_cluster.cc:1428] Setting key 387a6568863129ba47799521fc8fd984
I20260808 02:49:25.627605 27441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723: Bootstrap starting.
I20260808 02:49:25.630769 27447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:25.625069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:60575 (local address 127.0.0.1:38099)
0808 02:49:25.625889 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:25.625900 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:25.625940 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:49:25.627582 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:25.627607 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:25.627959 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:25.628256 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:25.629285 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:25.629298 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:25.629310 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:25.629325 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:25.629390 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:25.629409 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:25.629571 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:25.629677 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 02:49:25.629851 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":352,"threads_started":1}
I20260808 02:49:25.631074 27441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:25.632382 27441 log.cc:824] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:25.634678 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:25.635439 27441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723: No bootstrap required, opened a new log
I20260808 02:49:25.642071 27447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:25.639766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:54865 (local address 127.0.0.1:38099)
0808 02:49:25.639941 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:25.639947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:25.640292 (+ 345us) server_negotiation.cc:407] Connection header received
0808 02:49:25.640431 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:25.640440 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:25.640498 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:25.640608 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:25.641471 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:25.641477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:25.641483 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:25.641500 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:25.641550 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:25.641568 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:25.641691 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:25.641736 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 02:49:25.641795 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:49:25.644520 27441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019859189f434f8e87ebfc9945e61723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260808 02:49:25.644845 27441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:25.644917 27441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019859189f434f8e87ebfc9945e61723, State: Initialized, Role: FOLLOWER
I20260808 02:49:25.645419 27441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019859189f434f8e87ebfc9945e61723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260808 02:49:25.645638 27441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:25.645740 27441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:25.645864 27441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:25.648007 27441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019859189f434f8e87ebfc9945e61723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260808 02:49:25.648450 27441 leader_election.cc:302] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 019859189f434f8e87ebfc9945e61723; no voters:
I20260808 02:49:25.649001 27441 leader_election.cc:288] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:25.649121 27456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:25.649982 27456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [term 1 LEADER]: Becoming Leader. State: Replica: 019859189f434f8e87ebfc9945e61723, State: Running, Role: LEADER
I20260808 02:49:25.650512 27456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019859189f434f8e87ebfc9945e61723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260808 02:49:25.651013 27441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:25.653342 27457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "019859189f434f8e87ebfc9945e61723" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019859189f434f8e87ebfc9945e61723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } } }
I20260808 02:49:25.653649 27459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 019859189f434f8e87ebfc9945e61723. Latest consensus state: current_term: 1 leader_uuid: "019859189f434f8e87ebfc9945e61723" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019859189f434f8e87ebfc9945e61723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } } }
I20260808 02:49:25.653882 27457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:25.653906 27459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:25.654749 27470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:25.660653 27470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:25.669090 27470 catalog_manager.cc:1347] Generated new cluster ID: 3830df8bcc9d48168f17c5cc19463b14
I20260808 02:49:25.669198 27470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:25.686271 27470 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:25.686748 27470 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:25.693401 27470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 019859189f434f8e87ebfc9945e61723: Generated new TSK 0
I20260808 02:49:25.694190 27470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:25.703050 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27328
W20260808 02:49:25.704141 27431 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.199:60575
W20260808 02:49:25.704916 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27328: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:25.705083 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 27328
2026-08-08T02:49:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2101 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:25 dist-test-slave-k5rr krb5kdc[27492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:25 dist-test-slave-k5rr krb5kdc[27492](info): set up 2 sockets
Aug 08 02:49:25 dist-test-slave-k5rr krb5kdc[27492](info): commencing operation
krb5kdc: starting...
W20260808 02:49:26.935016 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:27 dist-test-slave-k5rr krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:27.257690 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35605
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:27.593096 27514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:27.593369 27514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:27.593502 27514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:27.600601 27514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:27.600699 27514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:27.600773 27514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:27.600859 27514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:27.600919 27514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:27.610534 27514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35605
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:27.611766 27514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:27.613044 27514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:27.624135 27519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:27.624228 27520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:27.624387 27523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:27.625402 27514 server_base.cc:1034] running on GCE node
I20260808 02:49:27.626101 27514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:27.627359 27514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:27.628552 27514 hybrid_clock.cc:648] HybridClock initialized: now 1786157367628513 us; error 29 us; skew 500 ppm
I20260808 02:49:27.628851 27514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:27 dist-test-slave-k5rr krb5kdc[27492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:27.635558 27514 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:27.636915 27514 webserver.cc:460] Webserver started at http://127.0.0.1:45045/ using document root <none> and password file <none>
I20260808 02:49:27.637573 27514 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:27.637676 27514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:27.637926 27514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:27.640820 27514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "b58a442570ef4242a89e510a22cca9e3"
format_stamp: "Formatted at 2026-08-08 02:49:27 on dist-test-slave-k5rr"
server_key: "36ca1b6bf734beab18f3ae682d755157"
server_key_iv: "19bc21be93706d14f9c22382f1f126ed"
server_key_version: "encryptionkey@0"
I20260808 02:49:27.641590 27514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "b58a442570ef4242a89e510a22cca9e3"
format_stamp: "Formatted at 2026-08-08 02:49:27 on dist-test-slave-k5rr"
server_key: "36ca1b6bf734beab18f3ae682d755157"
server_key_iv: "19bc21be93706d14f9c22382f1f126ed"
server_key_version: "encryptionkey@0"
I20260808 02:49:27.646476 27514 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 02:49:27.649979 27533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:27.651196 27514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:49:27.651376 27514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "b58a442570ef4242a89e510a22cca9e3"
format_stamp: "Formatted at 2026-08-08 02:49:27 on dist-test-slave-k5rr"
server_key: "36ca1b6bf734beab18f3ae682d755157"
server_key_iv: "19bc21be93706d14f9c22382f1f126ed"
server_key_version: "encryptionkey@0"
I20260808 02:49:27.651562 27514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:27.668771 27514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:27.672662 27514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:27.672926 27514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:27.690949 27514 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35605
I20260808 02:49:27.691006 27614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35605 every 8 connection(s)
I20260808 02:49:27.692328 27514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:27.696048 27615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:27.699237 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 27514
I20260808 02:49:27.699414 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:27.699849 26151 external_mini_cluster.cc:1428] Setting key 1ce03141dd1e948132d98442075f7b7d
I20260808 02:49:27.709483 27615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3: Bootstrap starting.
I20260808 02:49:27.713711 27615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:27.715036 27615 log.cc:824] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:27.718086 27615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3: No bootstrap required, opened a new log
Aug 08 02:49:27 dist-test-slave-k5rr krb5kdc[27492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:49:27.728256 27615 raft_consensus.cc:348] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58a442570ef4242a89e510a22cca9e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } }
I20260808 02:49:27.728739 27615 raft_consensus.cc:374] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:27.728859 27615 raft_consensus.cc:729] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58a442570ef4242a89e510a22cca9e3, State: Initialized, Role: FOLLOWER
I20260808 02:49:27.729290 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:27.701756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:41819 (local address 127.0.0.1:35605)
0808 02:49:27.702589 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:27.702605 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:27.702657 (+ 52us) server_negotiation.cc:407] Connection header received
0808 02:49:27.704863 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:27.704893 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:27.705469 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:27.705933 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:27.708494 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:27.711060 (+ 2566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:27.714436 (+ 3376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:27.715495 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:27.715711 (+ 216us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:27.720223 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:27.720233 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:27.720253 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:27.720297 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:27.724373 (+ 4076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:27.725253 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:27.725266 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:27.725275 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:27.725489 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:27.726021 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:27.726027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:27.726030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:27.726790 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:27.727113 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:27.727525 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 02:49:27.727866 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":328,"threads_started":1}
I20260808 02:49:27.729553 27615 consensus_queue.cc:260] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58a442570ef4242a89e510a22cca9e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } }
I20260808 02:49:27.729832 27615 raft_consensus.cc:388] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:27.729975 27615 raft_consensus.cc:482] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:27.730127 27615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:27.733085 27615 raft_consensus.cc:504] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58a442570ef4242a89e510a22cca9e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } }
I20260808 02:49:27.733546 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:27.733608 27615 leader_election.cc:302] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b58a442570ef4242a89e510a22cca9e3; no voters:
I20260808 02:49:27.734198 27615 leader_election.cc:288] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:27.734396 27621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:27.735976 27615 sys_catalog.cc:564] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:27.745543 27621 raft_consensus.cc:686] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [term 1 LEADER]: Becoming Leader. State: Replica: b58a442570ef4242a89e510a22cca9e3, State: Running, Role: LEADER
I20260808 02:49:27.746169 27621 consensus_queue.cc:237] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58a442570ef4242a89e510a22cca9e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } }
I20260808 02:49:27.748917 27622 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b58a442570ef4242a89e510a22cca9e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58a442570ef4242a89e510a22cca9e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } } }
I20260808 02:49:27.749325 27622 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:27.749305 27623 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b58a442570ef4242a89e510a22cca9e3. Latest consensus state: current_term: 1 leader_uuid: "b58a442570ef4242a89e510a22cca9e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58a442570ef4242a89e510a22cca9e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35605 } } }
I20260808 02:49:27.749606 27623 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3 [sys.catalog]: This master's current role is: LEADER
W20260808 02:49:27.758776 27652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:49:27.758905 27652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:49:27.759323 27650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:27.761608 27650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:27.763679 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:27.747115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:56619 (local address 127.0.0.1:35605)
0808 02:49:27.747296 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:27.747303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:27.747897 (+ 594us) server_negotiation.cc:407] Connection header received
0808 02:49:27.748100 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:27.748104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:27.748178 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:27.748265 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:27.750047 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:27.751290 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:27.753206 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:27.753886 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:27.753990 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:27.757272 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:27.757279 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:27.757285 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:27.757320 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:27.760680 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:27.761487 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:27.761492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:27.761494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:27.761618 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:27.762103 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:27.762109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:27.762113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:27.762464 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:27.762741 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:27.763183 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 02:49:27.763357 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:49:27.770814 27650 catalog_manager.cc:1347] Generated new cluster ID: 2e135275873b4c6abf531706591d7ace
I20260808 02:49:27.770927 27650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:27.792694 27650 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:27.794051 27650 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:27.802542 27650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b58a442570ef4242a89e510a22cca9e3: Generated new TSK 0
I20260808 02:49:27.803531 27650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:27.813230 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27514
I20260808 02:49:27.814247 27609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:27.903461 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 27514
W20260808 02:49:27.924811 27490 connection.cc:535] client connection to 127.0.0.1:35605 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35605): Connection reset by peer (error 104)
2026-08-08T02:49:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2201 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:28 dist-test-slave-k5rr krb5kdc[27664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:28 dist-test-slave-k5rr krb5kdc[27664](info): set up 2 sockets
Aug 08 02:49:28 dist-test-slave-k5rr krb5kdc[27664](info): commencing operation
krb5kdc: starting...
W20260808 02:49:29.159830 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:29 dist-test-slave-k5rr krb5kdc[27664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:29.487903 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:29.823442 27680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:29.823761 27680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:29.823904 27680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:29.830962 27680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:29.831086 27680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:29.831166 27680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:29.831259 27680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:29.831321 27680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:29.840915 27680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46593
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:29.842188 27680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:29.843526 27680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:29.854808 27693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:29.854846 27687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:29.854764 27695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:29.855664 27680 server_base.cc:1034] running on GCE node
I20260808 02:49:29.856408 27680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:29.857767 27680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:29.858973 27680 hybrid_clock.cc:648] HybridClock initialized: now 1786157369858922 us; error 37 us; skew 500 ppm
I20260808 02:49:29.859287 27680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:29 dist-test-slave-k5rr krb5kdc[27664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157369, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:29.862370 27680 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:29.863765 27680 webserver.cc:460] Webserver started at http://127.0.0.1:35929/ using document root <none> and password file <none>
I20260808 02:49:29.864396 27680 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:29.864523 27680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:29.864794 27680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:29.867717 27680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "6f9a6993132b4f59a7a4bba304794f02"
format_stamp: "Formatted at 2026-08-08 02:49:29 on dist-test-slave-k5rr"
server_key: "264c3af32ad9a16aeb87a5d13d25490b"
server_key_iv: "4cc5165add03a81282234033394f387e"
server_key_version: "encryptionkey@0"
I20260808 02:49:29.868438 27680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "6f9a6993132b4f59a7a4bba304794f02"
format_stamp: "Formatted at 2026-08-08 02:49:29 on dist-test-slave-k5rr"
server_key: "264c3af32ad9a16aeb87a5d13d25490b"
server_key_iv: "4cc5165add03a81282234033394f387e"
server_key_version: "encryptionkey@0"
I20260808 02:49:29.873301 27680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 02:49:29.876573 27703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:29.877879 27680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:49:29.878037 27680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "6f9a6993132b4f59a7a4bba304794f02"
format_stamp: "Formatted at 2026-08-08 02:49:29 on dist-test-slave-k5rr"
server_key: "264c3af32ad9a16aeb87a5d13d25490b"
server_key_iv: "4cc5165add03a81282234033394f387e"
server_key_version: "encryptionkey@0"
I20260808 02:49:29.878218 27680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:29.894685 27680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:29.898666 27680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:29.898936 27680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:29.916136 27680 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46593
I20260808 02:49:29.916183 27785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46593 every 8 connection(s)
I20260808 02:49:29.917590 27680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:29.919874 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 27680
I20260808 02:49:29.920059 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:29.920619 26151 external_mini_cluster.cc:1428] Setting key 0c6610d900f38b40c1ad8ffb170f6321
I20260808 02:49:29.921758 27787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:29.935003 27792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:29.922382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:40101 (local address 127.0.0.1:46593)
0808 02:49:29.923136 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:29.923151 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:29.923209 (+ 58us) server_negotiation.cc:407] Connection header received
0808 02:49:29.925115 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:29.925143 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:29.925673 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:29.926121 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:29.926615 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:29.928580 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:29.929875 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:29.930883 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:29.931056 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:29.932218 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:29.932230 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:29.932253 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:29.932979 (+ 726us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:29.933102 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:29.933132 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:29.933300 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:29.933530 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 02:49:29.933702 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":307,"threads_started":1}
I20260808 02:49:29.936466 27787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02: Bootstrap starting.
I20260808 02:49:29.938725 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:29.939874 27787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:29.941103 27787 log.cc:824] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:29.944139 27787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02: No bootstrap required, opened a new log
I20260808 02:49:29.950850 27792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:29.943976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:43509 (local address 127.0.0.1:46593)
0808 02:49:29.944167 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:29.944173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:29.944611 (+ 438us) server_negotiation.cc:407] Connection header received
0808 02:49:29.944771 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:29.944777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:29.944856 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:29.944975 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:29.945518 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:29.946663 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:29.948067 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:29.948702 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:29.948864 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:29.950030 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:29.950037 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:29.950042 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:29.950068 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:29.950129 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:29.950150 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:29.950276 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:29.950456 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 02:49:29.950515 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:49:29.953343 27787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9a6993132b4f59a7a4bba304794f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46593 } }
I20260808 02:49:29.953714 27787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:29.953788 27787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9a6993132b4f59a7a4bba304794f02, State: Initialized, Role: FOLLOWER
I20260808 02:49:29.954276 27787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9a6993132b4f59a7a4bba304794f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46593 } }
I20260808 02:49:29.954492 27787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:29.954664 27787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:29.954824 27787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:29.956990 27787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9a6993132b4f59a7a4bba304794f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46593 } }
I20260808 02:49:29.957535 27787 leader_election.cc:302] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f9a6993132b4f59a7a4bba304794f02; no voters:
I20260808 02:49:29.958089 27787 leader_election.cc:288] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:29.958236 27800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:29.959272 27800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [term 1 LEADER]: Becoming Leader. State: Replica: 6f9a6993132b4f59a7a4bba304794f02, State: Running, Role: LEADER
I20260808 02:49:29.960016 27800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9a6993132b4f59a7a4bba304794f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46593 } }
I20260808 02:49:29.960417 27787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:29.962837 27803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f9a6993132b4f59a7a4bba304794f02. Latest consensus state: current_term: 1 leader_uuid: "6f9a6993132b4f59a7a4bba304794f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9a6993132b4f59a7a4bba304794f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46593 } } }
I20260808 02:49:29.963193 27803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:29.963681 27802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f9a6993132b4f59a7a4bba304794f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9a6993132b4f59a7a4bba304794f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46593 } } }
I20260808 02:49:29.963992 27802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:29.964144 27811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:29.970276 27811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:29.980760 27811 catalog_manager.cc:1347] Generated new cluster ID: 04f0e57eb7e649879190b7de57346d4e
I20260808 02:49:29.980883 27811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:30.000499 27811 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:30.002415 27811 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:30.009178 27811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f9a6993132b4f59a7a4bba304794f02: Generated new TSK 0
I20260808 02:49:30.010020 27811 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:30.022996 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27680
W20260808 02:49:30.024282 27780 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.199:40101
W20260808 02:49:30.025107 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27680: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:30.025249 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 27680
W20260808 02:49:30.050235 27662 connection.cc:535] client connection to 127.0.0.1:46593 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46593): Connection reset by peer (error 104)
2026-08-08T02:49:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2120 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:30 dist-test-slave-k5rr krb5kdc[27830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:30 dist-test-slave-k5rr krb5kdc[27830](info): set up 2 sockets
Aug 08 02:49:30 dist-test-slave-k5rr krb5kdc[27830](info): commencing operation
krb5kdc: starting...
W20260808 02:49:31.273391 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.007s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:31 dist-test-slave-k5rr krb5kdc[27830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:49:31.596287 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:31.930871 27853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:31.931159 27853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:31.931259 27853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:31.938509 27853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:31.938616 27853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:31.938664 27853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:31.938724 27853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:31.938766 27853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:31.948491 27853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37715
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:31.949777 27853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:31.951052 27853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:31.962395 27862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:31.962458 27863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:31.962494 27866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:31.963317 27853 server_base.cc:1034] running on GCE node
I20260808 02:49:31.964104 27853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:31.965567 27853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:31.966778 27853 hybrid_clock.cc:648] HybridClock initialized: now 1786157371966727 us; error 41 us; skew 500 ppm
I20260808 02:49:31.967114 27853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:31 dist-test-slave-k5rr krb5kdc[27830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157371, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:31.970175 27853 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:49:31.971648 27853 webserver.cc:460] Webserver started at http://127.0.0.1:34419/ using document root <none> and password file <none>
I20260808 02:49:31.972280 27853 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:31.972386 27853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:31.972640 27853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:31.975574 27853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "969f387e267e47439a5fb51f035cbf20"
format_stamp: "Formatted at 2026-08-08 02:49:31 on dist-test-slave-k5rr"
server_key: "7645a4159f44c28bd49bcb67d3c02d39"
server_key_iv: "b612d1a5c11cc6a62939f5d4c226f8dc"
server_key_version: "encryptionkey@0"
I20260808 02:49:31.976310 27853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "969f387e267e47439a5fb51f035cbf20"
format_stamp: "Formatted at 2026-08-08 02:49:31 on dist-test-slave-k5rr"
server_key: "7645a4159f44c28bd49bcb67d3c02d39"
server_key_iv: "b612d1a5c11cc6a62939f5d4c226f8dc"
server_key_version: "encryptionkey@0"
I20260808 02:49:31.981438 27853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:49:31.984802 27875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:31.986212 27853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:49:31.986356 27853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "969f387e267e47439a5fb51f035cbf20"
format_stamp: "Formatted at 2026-08-08 02:49:31 on dist-test-slave-k5rr"
server_key: "7645a4159f44c28bd49bcb67d3c02d39"
server_key_iv: "b612d1a5c11cc6a62939f5d4c226f8dc"
server_key_version: "encryptionkey@0"
I20260808 02:49:31.986521 27853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:31.992157 27853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:31.996225 27853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:31.996469 27853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:32.014456 27853 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37715
I20260808 02:49:32.014503 27948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37715 every 8 connection(s)
I20260808 02:49:32.015852 27853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:32.016356 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 27853
I20260808 02:49:32.016518 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:32.017109 26151 external_mini_cluster.cc:1428] Setting key 5c6f8e3fb56ee8a1feb1e14df9ea0713
I20260808 02:49:32.019873 27949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:32.026356 27954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:32.019023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:57189 (local address 127.0.0.1:37715)
0808 02:49:32.019907 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:32.019922 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:32.020226 (+ 304us) server_negotiation.cc:407] Connection header received
0808 02:49:32.022522 (+ 2296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:32.022549 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:32.022990 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:32.023434 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:32.024401 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:32.024409 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:32.024427 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:32.024444 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:32.024538 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:32.024561 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:32.024777 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:32.024926 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 02:49:32.025148 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":358,"threads_started":1}
I20260808 02:49:32.030229 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:32.033963 27949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20: Bootstrap starting.
I20260808 02:49:32.037868 27949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:32.038440 27954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:32.035730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:44987 (local address 127.0.0.1:37715)
0808 02:49:32.035992 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:32.035997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:32.036440 (+ 443us) server_negotiation.cc:407] Connection header received
0808 02:49:32.036607 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:32.036611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:32.036658 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:32.036738 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:32.037720 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:32.037725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:32.037728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:32.037742 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:32.037816 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:32.037832 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:32.037921 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:32.038078 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 02:49:32.038126 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:49:32.039129 27949 log.cc:824] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:32.041848 27949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20: No bootstrap required, opened a new log
I20260808 02:49:32.049685 27949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969f387e267e47439a5fb51f035cbf20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37715 } }
I20260808 02:49:32.050036 27949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:32.050110 27949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 969f387e267e47439a5fb51f035cbf20, State: Initialized, Role: FOLLOWER
I20260808 02:49:32.050617 27949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969f387e267e47439a5fb51f035cbf20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37715 } }
I20260808 02:49:32.050812 27949 raft_consensus.cc:388] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:32.050910 27949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:32.051038 27949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:32.053181 27949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969f387e267e47439a5fb51f035cbf20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37715 } }
I20260808 02:49:32.053719 27949 leader_election.cc:302] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 969f387e267e47439a5fb51f035cbf20; no voters:
I20260808 02:49:32.054343 27949 leader_election.cc:288] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:32.054443 27962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:32.055097 27962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [term 1 LEADER]: Becoming Leader. State: Replica: 969f387e267e47439a5fb51f035cbf20, State: Running, Role: LEADER
I20260808 02:49:32.055703 27962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969f387e267e47439a5fb51f035cbf20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37715 } }
I20260808 02:49:32.056627 27949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:32.058530 27963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "969f387e267e47439a5fb51f035cbf20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969f387e267e47439a5fb51f035cbf20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37715 } } }
I20260808 02:49:32.058934 27963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:32.058825 27964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 969f387e267e47439a5fb51f035cbf20. Latest consensus state: current_term: 1 leader_uuid: "969f387e267e47439a5fb51f035cbf20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "969f387e267e47439a5fb51f035cbf20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37715 } } }
I20260808 02:49:32.059142 27964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:32.061093 27974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:32.065769 27974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:32.079684 27974 catalog_manager.cc:1347] Generated new cluster ID: 04277b1d157e4a90a52cc470ffb15724
I20260808 02:49:32.079891 27974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:32.088917 27974 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:32.089435 27974 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:32.102703 27974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 969f387e267e47439a5fb51f035cbf20: Generated new TSK 0
I20260808 02:49:32.103549 27974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:32.117707 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27853
W20260808 02:49:32.118762 27943 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.199:57189
W20260808 02:49:32.119432 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 27853: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:32.119560 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 27853
2026-08-08T02:49:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2086 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:32 dist-test-slave-k5rr krb5kdc[27990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:32 dist-test-slave-k5rr krb5kdc[27990](info): set up 2 sockets
Aug 08 02:49:32 dist-test-slave-k5rr krb5kdc[27990](info): commencing operation
krb5kdc: starting...
W20260808 02:49:33.366201 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.006s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:33 dist-test-slave-k5rr krb5kdc[27990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:49:33.697651 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:36523
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:35139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:36523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:34.041939 28015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:34.042243 28015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:34.042366 28015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:34.049465 28015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:34.049572 28015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:34.049654 28015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:34.049729 28015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:34.049796 28015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:34.059525 28015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:35139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:36523
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:36523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:34.060750 28015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:34.062047 28015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:34.073334 28023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:34.073285 28024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:34.073369 28026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:34.074576 28015 server_base.cc:1034] running on GCE node
I20260808 02:49:34.075413 28015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:34.076843 28015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:34.078063 28015 hybrid_clock.cc:648] HybridClock initialized: now 1786157374078006 us; error 42 us; skew 500 ppm
I20260808 02:49:34.078444 28015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:34 dist-test-slave-k5rr krb5kdc[27990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157374, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:34.085139 28015 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:49:34.086853 28015 webserver.cc:460] Webserver started at http://127.25.137.254:40365/ using document root <none> and password file <none>
I20260808 02:49:34.087585 28015 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:34.087718 28015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:34.088016 28015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:34.091161 28015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "b43af974b41047908dedd9f5cef20f7a"
format_stamp: "Formatted at 2026-08-08 02:49:34 on dist-test-slave-k5rr"
server_key: "26ec5fcc649e4707e7485e7d6113b68c"
server_key_iv: "8d5541d6b5b14437a218efd05a01d1b8"
server_key_version: "encryptionkey@0"
I20260808 02:49:34.091971 28015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "b43af974b41047908dedd9f5cef20f7a"
format_stamp: "Formatted at 2026-08-08 02:49:34 on dist-test-slave-k5rr"
server_key: "26ec5fcc649e4707e7485e7d6113b68c"
server_key_iv: "8d5541d6b5b14437a218efd05a01d1b8"
server_key_version: "encryptionkey@0"
I20260808 02:49:34.097342 28015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 02:49:34.100868 28037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:34.102214 28015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:49:34.102411 28015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "b43af974b41047908dedd9f5cef20f7a"
format_stamp: "Formatted at 2026-08-08 02:49:34 on dist-test-slave-k5rr"
server_key: "26ec5fcc649e4707e7485e7d6113b68c"
server_key_iv: "8d5541d6b5b14437a218efd05a01d1b8"
server_key_version: "encryptionkey@0"
I20260808 02:49:34.102602 28015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:34.128388 28015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:34.132638 28015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:34.132916 28015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:34.151283 28015 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:36523
I20260808 02:49:34.151326 28114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:36523 every 8 connection(s)
I20260808 02:49:34.152704 28015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:34.156404 28116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:34.160604 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 28015
I20260808 02:49:34.160774 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:34.161307 26151 external_mini_cluster.cc:1428] Setting key 0cc675e64eb46d2dcd6274574b399ca6
I20260808 02:49:34.167344 28116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a: Bootstrap starting.
I20260808 02:49:34.170887 28116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:34.172056 28116 log.cc:824] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:34.175501 28116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a: No bootstrap required, opened a new log
Aug 08 02:49:34 dist-test-slave-k5rr krb5kdc[27990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:34.184659 28116 raft_consensus.cc:348] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43af974b41047908dedd9f5cef20f7a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36523 } }
I20260808 02:49:34.185040 28116 raft_consensus.cc:374] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:34.185146 28116 raft_consensus.cc:729] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b43af974b41047908dedd9f5cef20f7a, State: Initialized, Role: FOLLOWER
I20260808 02:49:34.185225 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:34.163067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59295 (local address 127.25.137.254:36523)
0808 02:49:34.163853 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:34.163865 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:34.163907 (+ 42us) server_negotiation.cc:407] Connection header received
0808 02:49:34.165862 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:34.165885 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:34.166262 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:34.166587 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:34.168360 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:34.169746 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:34.171531 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:34.172270 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:34.172444 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:34.177437 (+ 4993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:34.177446 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:34.177461 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:34.177506 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:34.180785 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:34.181753 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:34.181760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:34.181766 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:34.182002 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:34.182451 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:34.182454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:34.182456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:34.183086 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:34.183336 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:34.183811 (+ 475us) server_negotiation.cc:299] Negotiation successful
0808 02:49:34.184100 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":363,"threads_started":1}
I20260808 02:49:34.185786 28116 consensus_queue.cc:260] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43af974b41047908dedd9f5cef20f7a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36523 } }
I20260808 02:49:34.186080 28116 raft_consensus.cc:388] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:34.186224 28116 raft_consensus.cc:482] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:34.186412 28116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:34.189067 28116 raft_consensus.cc:504] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43af974b41047908dedd9f5cef20f7a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36523 } }
I20260808 02:49:34.189283 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:34.189568 28116 leader_election.cc:302] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b43af974b41047908dedd9f5cef20f7a; no voters:
I20260808 02:49:34.190158 28116 leader_election.cc:288] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:34.190299 28126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:34.192019 28116 sys_catalog.cc:564] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:34.197196 28126 raft_consensus.cc:686] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [term 1 LEADER]: Becoming Leader. State: Replica: b43af974b41047908dedd9f5cef20f7a, State: Running, Role: LEADER
I20260808 02:49:34.198046 28126 consensus_queue.cc:237] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43af974b41047908dedd9f5cef20f7a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36523 } }
I20260808 02:49:34.201392 28129 sys_catalog.cc:455] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b43af974b41047908dedd9f5cef20f7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43af974b41047908dedd9f5cef20f7a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36523 } } }
I20260808 02:49:34.201783 28129 sys_catalog.cc:458] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:34.207327 28154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:34.207165 28130 sys_catalog.cc:455] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b43af974b41047908dedd9f5cef20f7a. Latest consensus state: current_term: 1 leader_uuid: "b43af974b41047908dedd9f5cef20f7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43af974b41047908dedd9f5cef20f7a" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 36523 } } }
I20260808 02:49:34.207494 28130 sys_catalog.cc:458] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:34.212572 28154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:34.220304 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:34.199700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35979 (local address 127.25.137.254:36523)
0808 02:49:34.199867 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:34.199873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:34.206230 (+ 6357us) server_negotiation.cc:407] Connection header received
0808 02:49:34.206541 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:34.206547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:34.206626 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:34.206731 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:34.208914 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:34.209756 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:34.211046 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:34.211540 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:34.211656 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:34.214172 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:34.214177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:34.214181 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:34.214212 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:34.217223 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:34.218182 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:34.218187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:34.218191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:34.218347 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:34.218848 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:34.218853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:34.218856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:34.219223 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:34.219472 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:34.219863 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 02:49:34.220028 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:49:34.223191 28154 catalog_manager.cc:1347] Generated new cluster ID: d2eca4d8ef9f4aec8b42c3ccfdbff2fd
I20260808 02:49:34.223295 28154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:34.266801 28154 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:34.268663 28154 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:34.289080 28154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b43af974b41047908dedd9f5cef20f7a: Generated new TSK 0
I20260808 02:49:34.289925 28154 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:34.299639 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28015
I20260808 02:49:34.300743 28110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:34.394411 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 28015
W20260808 02:49:34.418956 27987 connection.cc:535] client connection to 127.25.137.254:36523 recv error: Network error: failed to read from TLS socket (remote: 127.25.137.254:36523): Connection reset by peer (error 104)
2026-08-08T02:49:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2281 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:34 dist-test-slave-k5rr krb5kdc[28169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:34 dist-test-slave-k5rr krb5kdc[28169](info): set up 2 sockets
Aug 08 02:49:34 dist-test-slave-k5rr krb5kdc[28169](info): commencing operation
krb5kdc: starting...
W20260808 02:49:35.646678 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.001s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:35 dist-test-slave-k5rr krb5kdc[28169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:49:35.972584 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:40951
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:39855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:40951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:36.309358 28191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:36.309643 28191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:36.309787 28191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:36.316869 28191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:36.316972 28191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:36.317061 28191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:36.317143 28191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:36.317202 28191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:36.326923 28191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:39855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:40951
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:40951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:36.328161 28191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:36.329470 28191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:36.340643 28203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:36.340607 28204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:36.340569 28209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:36.341516 28191 server_base.cc:1034] running on GCE node
I20260808 02:49:36.342219 28191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:36.343550 28191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:36.344758 28191 hybrid_clock.cc:648] HybridClock initialized: now 1786157376344712 us; error 37 us; skew 500 ppm
I20260808 02:49:36.345155 28191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:36 dist-test-slave-k5rr krb5kdc[28169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157376, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:36.348192 28191 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:49:36.349677 28191 webserver.cc:460] Webserver started at http://127.25.137.254:45541/ using document root <none> and password file <none>
I20260808 02:49:36.350332 28191 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:36.350452 28191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:36.350728 28191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:36.353673 28191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "81902ec117a64b1f9661b68672e0317f"
format_stamp: "Formatted at 2026-08-08 02:49:36 on dist-test-slave-k5rr"
server_key: "f651b52af926d03b29ef24c2faecddc9"
server_key_iv: "4c66455842482a15a03e8b927f1f5ace"
server_key_version: "encryptionkey@0"
I20260808 02:49:36.354490 28191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "81902ec117a64b1f9661b68672e0317f"
format_stamp: "Formatted at 2026-08-08 02:49:36 on dist-test-slave-k5rr"
server_key: "f651b52af926d03b29ef24c2faecddc9"
server_key_iv: "4c66455842482a15a03e8b927f1f5ace"
server_key_version: "encryptionkey@0"
I20260808 02:49:36.359398 28191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:49:36.362612 28222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:36.363832 28191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:49:36.363996 28191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "81902ec117a64b1f9661b68672e0317f"
format_stamp: "Formatted at 2026-08-08 02:49:36 on dist-test-slave-k5rr"
server_key: "f651b52af926d03b29ef24c2faecddc9"
server_key_iv: "4c66455842482a15a03e8b927f1f5ace"
server_key_version: "encryptionkey@0"
I20260808 02:49:36.364161 28191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:36.384305 28191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:36.388098 28191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:36.388368 28191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:36.406452 28191 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:40951
I20260808 02:49:36.406490 28300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:40951 every 8 connection(s)
I20260808 02:49:36.407759 28191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:36.411382 28301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:36.414709 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 28191
I20260808 02:49:36.414881 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:36.415421 26151 external_mini_cluster.cc:1428] Setting key dc7b9f00d30cfa1103c50ee8d0c6f7e3
I20260808 02:49:36.423592 28301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f: Bootstrap starting.
I20260808 02:49:36.427037 28301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:36.428154 28301 log.cc:824] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:36.428505 28309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:36.417285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42647 (local address 127.25.137.254:40951)
0808 02:49:36.418060 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:36.418071 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:36.418112 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:49:36.419752 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:36.419775 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:36.420160 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:36.420501 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:36.421072 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:36.422417 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:36.424198 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:36.424837 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:36.424973 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:36.426214 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:36.426222 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:36.426234 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:36.426753 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:36.426826 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:36.426851 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:36.426976 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:36.427329 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 02:49:36.427514 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":364,"threads_started":1}
I20260808 02:49:36.431093 28301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f: No bootstrap required, opened a new log
I20260808 02:49:36.432334 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:36.440954 28301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81902ec117a64b1f9661b68672e0317f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40951 } }
I20260808 02:49:36.441409 28301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:36.441521 28301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81902ec117a64b1f9661b68672e0317f, State: Initialized, Role: FOLLOWER
I20260808 02:49:36.442183 28301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81902ec117a64b1f9661b68672e0317f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40951 } }
I20260808 02:49:36.442438 28301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:36.442582 28301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:36.442757 28301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:36.443951 28309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:36.437404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36891 (local address 127.25.137.254:40951)
0808 02:49:36.437567 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:36.437573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:36.438086 (+ 513us) server_negotiation.cc:407] Connection header received
0808 02:49:36.438237 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:36.438242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:36.438321 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:36.438421 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:36.438842 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:36.439982 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:36.441359 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:36.441981 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:36.442106 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:36.443182 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:36.443188 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:36.443193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:36.443214 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:36.443268 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:36.443296 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:36.443427 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:36.443568 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 02:49:36.443623 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 02:49:36.445494 28301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81902ec117a64b1f9661b68672e0317f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40951 } }
I20260808 02:49:36.445961 28301 leader_election.cc:302] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81902ec117a64b1f9661b68672e0317f; no voters:
I20260808 02:49:36.446478 28301 leader_election.cc:288] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:36.446637 28323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:36.447296 28323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [term 1 LEADER]: Becoming Leader. State: Replica: 81902ec117a64b1f9661b68672e0317f, State: Running, Role: LEADER
I20260808 02:49:36.447968 28323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81902ec117a64b1f9661b68672e0317f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40951 } }
I20260808 02:49:36.448804 28301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:36.450918 28325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81902ec117a64b1f9661b68672e0317f. Latest consensus state: current_term: 1 leader_uuid: "81902ec117a64b1f9661b68672e0317f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81902ec117a64b1f9661b68672e0317f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40951 } } }
I20260808 02:49:36.450942 28324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81902ec117a64b1f9661b68672e0317f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81902ec117a64b1f9661b68672e0317f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 40951 } } }
I20260808 02:49:36.451395 28325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:36.451395 28324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:36.453955 28332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:36.459059 28332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:36.467319 28332 catalog_manager.cc:1347] Generated new cluster ID: a5e9bcdb49934859ac6148d78e645fe4
I20260808 02:49:36.467435 28332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:36.480446 28332 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:36.481736 28332 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:36.489302 28332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81902ec117a64b1f9661b68672e0317f: Generated new TSK 0
I20260808 02:49:36.490093 28332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:36.497030 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28191
W20260808 02:49:36.498095 28291 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42647
W20260808 02:49:36.498852 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28191: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:36.498992 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 28191
W20260808 02:49:36.523061 28166 connection.cc:535] client connection to 127.25.137.254:40951 recv error: Network error: failed to read from TLS socket (remote: 127.25.137.254:40951): Connection reset by peer (error 104)
2026-08-08T02:49:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2103 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:36 dist-test-slave-k5rr krb5kdc[28361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:36 dist-test-slave-k5rr krb5kdc[28361](info): set up 2 sockets
Aug 08 02:49:36 dist-test-slave-k5rr krb5kdc[28361](info): commencing operation
krb5kdc: starting...
W20260808 02:49:37.739961 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:37 dist-test-slave-k5rr krb5kdc[28361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:49:38.064497 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:46869
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:46869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:38.401646 28379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:38.401935 28379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:38.402067 28379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:38.409211 28379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:38.409332 28379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:38.409403 28379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:38.409480 28379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:38.409538 28379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:38.419304 28379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:46869
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:46869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:38.420575 28379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:38.421883 28379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:38.433069 28385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:38.433111 28389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:38.433147 28386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:38.433830 28379 server_base.cc:1034] running on GCE node
I20260808 02:49:38.434553 28379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:38.435958 28379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:38.437172 28379 hybrid_clock.cc:648] HybridClock initialized: now 1786157378437121 us; error 40 us; skew 500 ppm
I20260808 02:49:38.437505 28379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:38 dist-test-slave-k5rr krb5kdc[28361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157378, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:38.443409 28379 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:49:38.444877 28379 webserver.cc:460] Webserver started at http://127.25.137.254:41461/ using document root <none> and password file <none>
I20260808 02:49:38.445564 28379 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:38.445683 28379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:38.445986 28379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:38.449002 28379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "eb1498efc8a145968e026d6942a73a8f"
format_stamp: "Formatted at 2026-08-08 02:49:38 on dist-test-slave-k5rr"
server_key: "8ba7dec80467b427934214f9c14a5690"
server_key_iv: "5caf8cbae3070f092006a113d54f1525"
server_key_version: "encryptionkey@0"
I20260808 02:49:38.449916 28379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "eb1498efc8a145968e026d6942a73a8f"
format_stamp: "Formatted at 2026-08-08 02:49:38 on dist-test-slave-k5rr"
server_key: "8ba7dec80467b427934214f9c14a5690"
server_key_iv: "5caf8cbae3070f092006a113d54f1525"
server_key_version: "encryptionkey@0"
I20260808 02:49:38.454998 28379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 02:49:38.458467 28400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:38.459771 28379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:49:38.459983 28379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "eb1498efc8a145968e026d6942a73a8f"
format_stamp: "Formatted at 2026-08-08 02:49:38 on dist-test-slave-k5rr"
server_key: "8ba7dec80467b427934214f9c14a5690"
server_key_iv: "5caf8cbae3070f092006a113d54f1525"
server_key_version: "encryptionkey@0"
I20260808 02:49:38.460172 28379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:38.465767 28379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:38.469651 28379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:38.469929 28379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:38.487339 28379 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:46869
I20260808 02:49:38.487381 28484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:46869 every 8 connection(s)
I20260808 02:49:38.488662 28379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:38.492319 28485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:38.496279 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 28379
I20260808 02:49:38.496459 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:38.496990 26151 external_mini_cluster.cc:1428] Setting key a18df4e22e4d9e0db9683ed3eb607cba
I20260808 02:49:38.504505 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:38.498736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51541 (local address 127.25.137.254:46869)
0808 02:49:38.499558 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:38.499569 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:38.499612 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:49:38.501232 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:38.501255 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:38.501604 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:38.501927 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:38.502998 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:38.503004 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:38.503015 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:38.503029 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:38.503087 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:38.503106 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:38.503273 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:38.503386 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 02:49:38.503551 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":359,"threads_started":1}
I20260808 02:49:38.505182 28485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f: Bootstrap starting.
I20260808 02:49:38.508101 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:38.508255 28485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:38.509454 28485 log.cc:824] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:38.512622 28485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f: No bootstrap required, opened a new log
I20260808 02:49:38.517103 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:38.513812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46929 (local address 127.25.137.254:46869)
0808 02:49:38.514175 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:38.514181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:38.514433 (+ 252us) server_negotiation.cc:407] Connection header received
0808 02:49:38.514709 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:38.514715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:38.514779 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:38.514861 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:38.516109 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:38.516115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:38.516120 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:38.516159 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:38.516207 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:38.516227 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:38.516356 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:38.516650 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 02:49:38.516795 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
I20260808 02:49:38.522759 28485 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb1498efc8a145968e026d6942a73a8f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46869 } }
I20260808 02:49:38.523264 28485 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:38.523376 28485 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb1498efc8a145968e026d6942a73a8f, State: Initialized, Role: FOLLOWER
I20260808 02:49:38.524025 28485 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb1498efc8a145968e026d6942a73a8f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46869 } }
I20260808 02:49:38.524377 28485 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:38.524562 28485 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:38.524739 28485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:38.527352 28485 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb1498efc8a145968e026d6942a73a8f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46869 } }
I20260808 02:49:38.527944 28485 leader_election.cc:302] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb1498efc8a145968e026d6942a73a8f; no voters:
I20260808 02:49:38.528654 28485 leader_election.cc:288] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:38.528793 28497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:38.534610 28485 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:38.536103 28497 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [term 1 LEADER]: Becoming Leader. State: Replica: eb1498efc8a145968e026d6942a73a8f, State: Running, Role: LEADER
I20260808 02:49:38.536676 28497 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb1498efc8a145968e026d6942a73a8f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46869 } }
I20260808 02:49:38.539585 28502 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb1498efc8a145968e026d6942a73a8f. Latest consensus state: current_term: 1 leader_uuid: "eb1498efc8a145968e026d6942a73a8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb1498efc8a145968e026d6942a73a8f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46869 } } }
I20260808 02:49:38.539609 28498 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb1498efc8a145968e026d6942a73a8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb1498efc8a145968e026d6942a73a8f" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 46869 } } }
I20260808 02:49:38.540033 28498 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:38.541826 28502 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:38.542094 28516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:38.547077 28516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:38.558692 28516 catalog_manager.cc:1347] Generated new cluster ID: 45c88913ad1343e3aa25b4696ac01b59
I20260808 02:49:38.558831 28516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:38.579427 28516 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:38.579890 28516 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:38.585230 28516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb1498efc8a145968e026d6942a73a8f: Generated new TSK 0
I20260808 02:49:38.585973 28516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:38.599403 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28379
W20260808 02:49:38.600502 28477 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51541
W20260808 02:49:38.601166 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28379: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:38.601321 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 28379
2026-08-08T02:49:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2106 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:38 dist-test-slave-k5rr krb5kdc[28529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:38 dist-test-slave-k5rr krb5kdc[28529](info): set up 2 sockets
Aug 08 02:49:38 dist-test-slave-k5rr krb5kdc[28529](info): commencing operation
krb5kdc: starting...
W20260808 02:49:39.871078 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:40 dist-test-slave-k5rr krb5kdc[28529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:49:40.194520 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:37201
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:42853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:37201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:40.531102 28553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:40.531384 28553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:40.531538 28553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:40.538673 28553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:40.538782 28553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:40.538856 28553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:40.538936 28553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:40.538996 28553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:40.549383 28553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:42853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:37201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:37201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:40.550622 28553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:40.551955 28553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:40.563594 28561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:40.563622 28563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:40.563714 28560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:40.564603 28553 server_base.cc:1034] running on GCE node
I20260808 02:49:40.565493 28553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:40.566959 28553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:40.568189 28553 hybrid_clock.cc:648] HybridClock initialized: now 1786157380568134 us; error 47 us; skew 500 ppm
I20260808 02:49:40.568547 28553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:40 dist-test-slave-k5rr krb5kdc[28529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157380, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:40.571593 28553 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:49:40.573081 28553 webserver.cc:460] Webserver started at http://127.25.137.254:46487/ using document root <none> and password file <none>
I20260808 02:49:40.573724 28553 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:40.573830 28553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:40.574083 28553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:40.576960 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "4e60a8ef2a2b4295a5fa1542bca4b96c"
format_stamp: "Formatted at 2026-08-08 02:49:40 on dist-test-slave-k5rr"
server_key: "e92168353825e7f341b9f5d581b0aaaf"
server_key_iv: "8b9cad3ffaaba24b69538ad4ec4476c7"
server_key_version: "encryptionkey@0"
I20260808 02:49:40.577754 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "4e60a8ef2a2b4295a5fa1542bca4b96c"
format_stamp: "Formatted at 2026-08-08 02:49:40 on dist-test-slave-k5rr"
server_key: "e92168353825e7f341b9f5d581b0aaaf"
server_key_iv: "8b9cad3ffaaba24b69538ad4ec4476c7"
server_key_version: "encryptionkey@0"
I20260808 02:49:40.582842 28553 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:49:40.586246 28576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:40.587644 28553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 02:49:40.587869 28553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "4e60a8ef2a2b4295a5fa1542bca4b96c"
format_stamp: "Formatted at 2026-08-08 02:49:40 on dist-test-slave-k5rr"
server_key: "e92168353825e7f341b9f5d581b0aaaf"
server_key_iv: "8b9cad3ffaaba24b69538ad4ec4476c7"
server_key_version: "encryptionkey@0"
I20260808 02:49:40.588060 28553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:40.606178 28553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:40.610193 28553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:40.610460 28553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:40.628432 28553 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:37201
I20260808 02:49:40.628473 28656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:37201 every 8 connection(s)
I20260808 02:49:40.629832 28553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:40.633589 28658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:40.636704 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 28553
I20260808 02:49:40.636915 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:40.637470 26151 external_mini_cluster.cc:1428] Setting key c30b421f120fcdd96b93dfffab9a8085
I20260808 02:49:40.645886 28658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c: Bootstrap starting.
I20260808 02:49:40.649507 28658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:40.650748 28658 log.cc:824] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:49:40 dist-test-slave-k5rr krb5kdc[28529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:40.654004 28658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c: No bootstrap required, opened a new log
I20260808 02:49:40.660753 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:40.639283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55713 (local address 127.25.137.254:37201)
0808 02:49:40.640039 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:40.640051 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:40.640090 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:49:40.641792 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:40.641815 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:40.642203 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:40.642521 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:40.644430 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:40.645794 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:40.647503 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:40.648252 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:40.648436 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:40.653420 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:40.653428 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:40.653442 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:40.653478 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:40.656686 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:40.657509 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:40.657515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:40.657522 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:40.657690 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:40.658082 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:40.658086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:40.658087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:40.658640 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:40.658881 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:40.659354 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 02:49:40.659720 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":340,"threads_started":1}
I20260808 02:49:40.662755 28658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e60a8ef2a2b4295a5fa1542bca4b96c" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37201 } }
I20260808 02:49:40.663193 28658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:40.663312 28658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e60a8ef2a2b4295a5fa1542bca4b96c, State: Initialized, Role: FOLLOWER
I20260808 02:49:40.663893 28658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e60a8ef2a2b4295a5fa1542bca4b96c" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37201 } }
I20260808 02:49:40.664147 28658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:40.664294 28658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:40.664464 28658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:40.664660 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:40.666661 28658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e60a8ef2a2b4295a5fa1542bca4b96c" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37201 } }
I20260808 02:49:40.667126 28658 leader_election.cc:302] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e60a8ef2a2b4295a5fa1542bca4b96c; no voters:
I20260808 02:49:40.667784 28658 leader_election.cc:288] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:40.667959 28669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:40.669624 28669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [term 1 LEADER]: Becoming Leader. State: Replica: 4e60a8ef2a2b4295a5fa1542bca4b96c, State: Running, Role: LEADER
I20260808 02:49:40.670190 28658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:40.670123 28669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e60a8ef2a2b4295a5fa1542bca4b96c" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37201 } }
I20260808 02:49:40.677721 28672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e60a8ef2a2b4295a5fa1542bca4b96c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e60a8ef2a2b4295a5fa1542bca4b96c" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37201 } } }
I20260808 02:49:40.678028 28672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:40.678460 28674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e60a8ef2a2b4295a5fa1542bca4b96c. Latest consensus state: current_term: 1 leader_uuid: "4e60a8ef2a2b4295a5fa1542bca4b96c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e60a8ef2a2b4295a5fa1542bca4b96c" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 37201 } } }
I20260808 02:49:40.678771 28674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:40.678884 28700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:40.684154 28700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:40.687218 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:40.669350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60125 (local address 127.25.137.254:37201)
0808 02:49:40.671198 (+ 1848us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:40.671203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:40.671222 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:49:40.671286 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:40.671294 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:40.671375 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:40.671473 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:40.673505 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:40.674744 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:40.676621 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:40.677294 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:40.677422 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:40.680247 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:40.680253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:40.680258 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:40.680303 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:40.684090 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:40.684972 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:40.684977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:40.684980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:40.685162 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:40.685579 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:40.685584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:40.685587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:40.685997 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:40.686240 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:40.686624 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 02:49:40.686860 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1730}
I20260808 02:49:40.693372 28700 catalog_manager.cc:1347] Generated new cluster ID: 244200fdb74c4b0498f94b1a2407372c
I20260808 02:49:40.693454 28700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:40.721414 28700 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:40.723315 28700 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:40.732486 28700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e60a8ef2a2b4295a5fa1542bca4b96c: Generated new TSK 0
I20260808 02:49:40.733338 28700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:40.739648 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28553
I20260808 02:49:40.740721 28649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:40.831710 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 28553
2026-08-08T02:49:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2223 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:40 dist-test-slave-k5rr krb5kdc[28711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:40 dist-test-slave-k5rr krb5kdc[28711](info): set up 2 sockets
Aug 08 02:49:40 dist-test-slave-k5rr krb5kdc[28711](info): commencing operation
krb5kdc: starting...
W20260808 02:49:42.063858 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:42 dist-test-slave-k5rr krb5kdc[28711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:49:42.394244 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45365
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:45365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:42.733764 28732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:42.734056 28732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:42.734176 28732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:42.741287 28732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:42.741391 28732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:42.741465 28732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:42.741541 28732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:42.741600 28732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:42.753306 28732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:45365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:45365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:42.755450 28732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:42.758656 28732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:42.770105 28740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:42.770152 28739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:42.770186 28742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:42.771039 28732 server_base.cc:1034] running on GCE node
I20260808 02:49:42.771943 28732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:42.773485 28732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:42.774730 28732 hybrid_clock.cc:648] HybridClock initialized: now 1786157382774674 us; error 43 us; skew 500 ppm
I20260808 02:49:42.775158 28732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:42 dist-test-slave-k5rr krb5kdc[28711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157382, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:42.778317 28732 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:49:42.779768 28732 webserver.cc:460] Webserver started at http://127.25.137.254:38229/ using document root <none> and password file <none>
I20260808 02:49:42.780436 28732 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:42.780555 28732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:42.780846 28732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:42.783804 28732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "3bd73bf9e690448d888e7c9f682fc1a4"
format_stamp: "Formatted at 2026-08-08 02:49:42 on dist-test-slave-k5rr"
server_key: "3deceee58e5c3964555b5f6aba7ef05b"
server_key_iv: "a941e90c54e7c611ac88e5211a2ea353"
server_key_version: "encryptionkey@0"
I20260808 02:49:42.784548 28732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "3bd73bf9e690448d888e7c9f682fc1a4"
format_stamp: "Formatted at 2026-08-08 02:49:42 on dist-test-slave-k5rr"
server_key: "3deceee58e5c3964555b5f6aba7ef05b"
server_key_iv: "a941e90c54e7c611ac88e5211a2ea353"
server_key_version: "encryptionkey@0"
I20260808 02:49:42.789458 28732 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:49:42.792755 28753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:42.793985 28732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:49:42.794142 28732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "3bd73bf9e690448d888e7c9f682fc1a4"
format_stamp: "Formatted at 2026-08-08 02:49:42 on dist-test-slave-k5rr"
server_key: "3deceee58e5c3964555b5f6aba7ef05b"
server_key_iv: "a941e90c54e7c611ac88e5211a2ea353"
server_key_version: "encryptionkey@0"
I20260808 02:49:42.794308 28732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:42.819866 28732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:42.823949 28732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:42.824224 28732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:42.841893 28732 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:45365
I20260808 02:49:42.841929 28843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:45365 every 8 connection(s)
I20260808 02:49:42.843243 28732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:42.846949 28844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:42.847196 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 28732
I20260808 02:49:42.847391 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:42.847975 26151 external_mini_cluster.cc:1428] Setting key 17c6c4cfa476134e7f7175409054da71
I20260808 02:49:42.859822 28844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4: Bootstrap starting.
I20260808 02:49:42.860589 28848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:42.849827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50301 (local address 127.25.137.254:45365)
0808 02:49:42.850594 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:42.850605 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:42.850646 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:49:42.852167 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:42.852188 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:42.852571 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:42.852941 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:42.853549 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:42.854859 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:42.856133 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:42.856853 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:42.857049 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:42.858265 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:42.858273 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:42.858287 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:42.858815 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:42.858895 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:42.858920 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:42.859028 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:42.859245 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 02:49:42.859497 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":335,"threads_started":1}
I20260808 02:49:42.863034 28844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:42.864147 28844 log.cc:824] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:42.864282 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:42.866784 28844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4: No bootstrap required, opened a new log
I20260808 02:49:42.874899 28844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd73bf9e690448d888e7c9f682fc1a4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45365 } }
I20260808 02:49:42.875288 28844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:42.875401 28844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd73bf9e690448d888e7c9f682fc1a4, State: Initialized, Role: FOLLOWER
I20260808 02:49:42.875581 28848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:42.869110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53197 (local address 127.25.137.254:45365)
0808 02:49:42.869314 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:42.869320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:42.869612 (+ 292us) server_negotiation.cc:407] Connection header received
0808 02:49:42.869846 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:42.869851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:42.869924 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:42.870016 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:42.870619 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:42.871439 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:42.872865 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:42.873420 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:42.873569 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:42.874700 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:42.874706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:42.874709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:42.874727 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:42.874819 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:42.874837 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:42.874936 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:42.875073 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 02:49:42.875179 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 02:49:42.876003 28844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd73bf9e690448d888e7c9f682fc1a4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45365 } }
I20260808 02:49:42.876242 28844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:42.876385 28844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:42.876546 28844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:42.878815 28844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd73bf9e690448d888e7c9f682fc1a4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45365 } }
I20260808 02:49:42.879298 28844 leader_election.cc:302] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bd73bf9e690448d888e7c9f682fc1a4; no voters:
I20260808 02:49:42.879868 28844 leader_election.cc:288] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:42.880012 28860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:42.881855 28844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:42.885293 28860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3bd73bf9e690448d888e7c9f682fc1a4, State: Running, Role: LEADER
I20260808 02:49:42.885957 28860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd73bf9e690448d888e7c9f682fc1a4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45365 } }
I20260808 02:49:42.889112 28861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bd73bf9e690448d888e7c9f682fc1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd73bf9e690448d888e7c9f682fc1a4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45365 } } }
I20260808 02:49:42.889550 28862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bd73bf9e690448d888e7c9f682fc1a4. Latest consensus state: current_term: 1 leader_uuid: "3bd73bf9e690448d888e7c9f682fc1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd73bf9e690448d888e7c9f682fc1a4" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 45365 } } }
I20260808 02:49:42.889786 28862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:42.889797 28861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4 [sys.catalog]: This master's current role is: LEADER
W20260808 02:49:42.893756 28876 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:49:42.893870 28876 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:49:42.893996 28879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:42.895700 28879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:42.903512 28879 catalog_manager.cc:1347] Generated new cluster ID: fb57278bbbd241fb8565b48b912895e2
I20260808 02:49:42.903623 28879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:42.915853 28879 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:42.917655 28879 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:42.930357 28879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bd73bf9e690448d888e7c9f682fc1a4: Generated new TSK 0
I20260808 02:49:42.931386 28879 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:42.939685 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28732
W20260808 02:49:42.940799 28837 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50301
W20260808 02:49:42.941542 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28732: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:42.941671 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 28732
2026-08-08T02:49:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2109 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:43 dist-test-slave-k5rr krb5kdc[28887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:43 dist-test-slave-k5rr krb5kdc[28887](info): set up 2 sockets
Aug 08 02:49:43 dist-test-slave-k5rr krb5kdc[28887](info): commencing operation
krb5kdc: starting...
W20260808 02:49:44.200843 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:44 dist-test-slave-k5rr krb5kdc[28887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:49:44.553903 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:38201
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:38201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:44.897843 28910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:44.898123 28910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:44.898224 28910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:44.905345 28910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:44.905457 28910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:44.905511 28910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:44.905571 28910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:44.905615 28910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:44.915715 28910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:38201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:38201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:44.916941 28910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:44.918248 28910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:44.929332 28916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:44.929404 28915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:44.929404 28919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:44.930135 28910 server_base.cc:1034] running on GCE node
I20260808 02:49:44.930869 28910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:44.932209 28910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:44.933427 28910 hybrid_clock.cc:648] HybridClock initialized: now 1786157384933383 us; error 36 us; skew 500 ppm
I20260808 02:49:44.933777 28910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:44 dist-test-slave-k5rr krb5kdc[28887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157384, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:44.939606 28910 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:49:44.941182 28910 webserver.cc:460] Webserver started at http://127.25.137.254:41331/ using document root <none> and password file <none>
I20260808 02:49:44.941854 28910 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:44.941962 28910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:44.942226 28910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:44.945314 28910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "ca979846348e4bebbe517593dce4f670"
format_stamp: "Formatted at 2026-08-08 02:49:44 on dist-test-slave-k5rr"
server_key: "848ebfc8557d7110c2324fbdd96367ed"
server_key_iv: "5b0c90efda19dedb801f6fb950381857"
server_key_version: "encryptionkey@0"
I20260808 02:49:44.946053 28910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "ca979846348e4bebbe517593dce4f670"
format_stamp: "Formatted at 2026-08-08 02:49:44 on dist-test-slave-k5rr"
server_key: "848ebfc8557d7110c2324fbdd96367ed"
server_key_iv: "5b0c90efda19dedb801f6fb950381857"
server_key_version: "encryptionkey@0"
I20260808 02:49:44.951437 28910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:49:44.954952 28931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:44.956353 28910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:49:44.956493 28910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "ca979846348e4bebbe517593dce4f670"
format_stamp: "Formatted at 2026-08-08 02:49:44 on dist-test-slave-k5rr"
server_key: "848ebfc8557d7110c2324fbdd96367ed"
server_key_iv: "5b0c90efda19dedb801f6fb950381857"
server_key_version: "encryptionkey@0"
I20260808 02:49:44.956673 28910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:44.962280 28910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:44.966609 28910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:44.966873 28910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:44.984485 28910 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:38201
I20260808 02:49:44.984516 29007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:38201 every 8 connection(s)
I20260808 02:49:44.985886 28910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:44.988193 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 28910
I20260808 02:49:44.988351 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:44.988765 26151 external_mini_cluster.cc:1428] Setting key aea495e27f575b3ae8186597f3494dc7
I20260808 02:49:44.989876 29008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:44.997146 29011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:44.990489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35811 (local address 127.25.137.254:38201)
0808 02:49:44.991325 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:44.991340 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:44.991391 (+ 51us) server_negotiation.cc:407] Connection header received
0808 02:49:44.993308 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:44.993336 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:44.993772 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:44.994210 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:44.995191 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:44.995199 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:44.995216 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:44.995233 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:44.995329 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:44.995355 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:44.995566 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:44.995699 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 02:49:44.995911 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":324,"threads_started":1}
I20260808 02:49:45.000748 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:45.004406 29008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670: Bootstrap starting.
I20260808 02:49:45.008311 29011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:45.006020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54891 (local address 127.25.137.254:38201)
0808 02:49:45.006201 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:45.006207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:45.006516 (+ 309us) server_negotiation.cc:407] Connection header received
0808 02:49:45.006662 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:45.006667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:45.006729 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:45.006851 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:45.007719 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:45.007724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:45.007728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:49:45.007745 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:45.007791 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:49:45.007811 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:45.007924 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:45.007971 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 02:49:45.008025 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:49:45.008378 29008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:45.009655 29008 log.cc:824] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:45.012662 29008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670: No bootstrap required, opened a new log
I20260808 02:49:45.020169 29008 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca979846348e4bebbe517593dce4f670" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38201 } }
I20260808 02:49:45.020550 29008 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:49:45.020624 29008 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca979846348e4bebbe517593dce4f670, State: Initialized, Role: FOLLOWER
I20260808 02:49:45.021160 29008 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca979846348e4bebbe517593dce4f670" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38201 } }
I20260808 02:49:45.021373 29008 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:49:45.021489 29008 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:49:45.021620 29008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:45.024101 29008 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca979846348e4bebbe517593dce4f670" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38201 } }
I20260808 02:49:45.024549 29008 leader_election.cc:302] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca979846348e4bebbe517593dce4f670; no voters:
I20260808 02:49:45.025101 29008 leader_election.cc:288] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:49:45.025252 29021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:45.025852 29021 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [term 1 LEADER]: Becoming Leader. State: Replica: ca979846348e4bebbe517593dce4f670, State: Running, Role: LEADER
I20260808 02:49:45.026428 29021 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca979846348e4bebbe517593dce4f670" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38201 } }
I20260808 02:49:45.027079 29008 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:45.029151 29022 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca979846348e4bebbe517593dce4f670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca979846348e4bebbe517593dce4f670" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38201 } } }
I20260808 02:49:45.032207 29022 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:45.032089 29023 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca979846348e4bebbe517593dce4f670. Latest consensus state: current_term: 1 leader_uuid: "ca979846348e4bebbe517593dce4f670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca979846348e4bebbe517593dce4f670" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 38201 } } }
I20260808 02:49:45.033128 29023 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:45.033723 29030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:45.038897 29030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:45.048303 29030 catalog_manager.cc:1347] Generated new cluster ID: add2811feb03415082677b666d61f9dd
I20260808 02:49:45.048434 29030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:45.062924 29030 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:45.063329 29030 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:45.069356 29030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca979846348e4bebbe517593dce4f670: Generated new TSK 0
I20260808 02:49:45.070160 29030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:45.076521 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28910
W20260808 02:49:45.077797 29002 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35811
W20260808 02:49:45.078574 26151 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 28910: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:49:45.078708 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 28910
2026-08-08T02:49:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2141 ms)
[----------] 18 tests from AuthTokenIssuingTest (38806 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:45 dist-test-slave-k5rr krb5kdc[29052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:45 dist-test-slave-k5rr krb5kdc[29052](info): set up 2 sockets
Aug 08 02:49:45 dist-test-slave-k5rr krb5kdc[29052](info): commencing operation
krb5kdc: starting...
W20260808 02:49:46.324111 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:46 dist-test-slave-k5rr krb5kdc[29052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:49:46.653949 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:39393
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:39393,127.25.137.253:35317,127.25.137.252:43275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:46.992221 29076 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:49:46.992537 29076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:46.992609 29076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:46.992697 29076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:46.999894 29076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:46.999982 29076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:47.000051 29076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:47.000104 29076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:47.000168 29076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:47.010087 29076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:39393,127.25.137.253:35317,127.25.137.252:43275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:39393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:47.011358 29076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:47.012676 29076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:47.024209 29084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:47.024287 29083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:47.024287 29088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:47.024955 29076 server_base.cc:1034] running on GCE node
I20260808 02:49:47.025722 29076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:47.027081 29076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:47.028321 29076 hybrid_clock.cc:648] HybridClock initialized: now 1786157387028237 us; error 42 us; skew 500 ppm
I20260808 02:49:47.028636 29076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:47 dist-test-slave-k5rr krb5kdc[29052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157387, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:47.031730 29076 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:49:47.033107 29076 webserver.cc:460] Webserver started at http://127.25.137.254:33801/ using document root <none> and password file <none>
I20260808 02:49:47.033764 29076 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:47.033870 29076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:47.034173 29076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:47.037122 29076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "3a199450f9c147e1a1f5ccc24582b356"
format_stamp: "Formatted at 2026-08-08 02:49:47 on dist-test-slave-k5rr"
server_key: "06a42d82f905aa1d4938a2daec2ba42f"
server_key_iv: "f55b0732022607b81e9d444f73ed9ded"
server_key_version: "encryptionkey@0"
I20260808 02:49:47.037909 29076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "3a199450f9c147e1a1f5ccc24582b356"
format_stamp: "Formatted at 2026-08-08 02:49:47 on dist-test-slave-k5rr"
server_key: "06a42d82f905aa1d4938a2daec2ba42f"
server_key_iv: "f55b0732022607b81e9d444f73ed9ded"
server_key_version: "encryptionkey@0"
I20260808 02:49:47.042809 29076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:49:47.046142 29100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:47.047426 29076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:49:47.047619 29076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "3a199450f9c147e1a1f5ccc24582b356"
format_stamp: "Formatted at 2026-08-08 02:49:47 on dist-test-slave-k5rr"
server_key: "06a42d82f905aa1d4938a2daec2ba42f"
server_key_iv: "f55b0732022607b81e9d444f73ed9ded"
server_key_version: "encryptionkey@0"
I20260808 02:49:47.047848 29076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:47.059508 29076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:47.063676 29076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:47.063956 29076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:47.082362 29076 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:39393
I20260808 02:49:47.082381 29170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:39393 every 8 connection(s)
I20260808 02:49:47.083748 29076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:47.087035 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 29076
I20260808 02:49:47.087244 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:47.087916 29172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:47.088205 26151 external_mini_cluster.cc:1428] Setting key 2c8e07a8d32f8037631288f0c6018e05
I20260808 02:49:47.096771 29172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:49:47 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157387, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:47.121474 29175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.100449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57060 (local address 127.25.137.254:39393)
0808 02:49:47.101351 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:47.101370 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:47.101877 (+ 507us) server_negotiation.cc:407] Connection header received
0808 02:49:47.103897 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:47.103907 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:47.104208 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:47.104415 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:47.105938 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.107158 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.109153 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.109707 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.109875 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:47.113870 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:47.113880 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:47.113892 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:47.113924 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:47.116994 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:47.118098 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:47.118105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:47.118107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:47.118255 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:47.118692 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:47.118695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:47.118697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:47.119201 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:47.119361 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:47.120016 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 02:49:47.120269 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":373,"threads_started":1}
I20260808 02:49:47.121510 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.100449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:39393 (local address 127.0.0.1:57060)
0808 02:49:47.101695 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.101736 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:47.103462 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:47.104460 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:47.104465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:47.105244 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:47.105700 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:47.105722 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.107350 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.107355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:47.108959 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:47.108974 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.109956 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.109963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:47.110129 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:47.110797 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:47.110831 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:47.113562 (+ 2731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:47.117144 (+ 3582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:47.117157 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:47.117169 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:47.117961 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:47.118364 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:47.118368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:47.118371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:47.118578 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:47.119350 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:47.119358 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:47.119620 (+ 262us) client_negotiation.cc:769] Sending connection context
0808 02:49:47.119877 (+ 257us) client_negotiation.cc:240] Negotiation successful
0808 02:49:47.120478 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"mutex_wait_us":87,"spinlock_wait_cycles":63872,"thread_start_us":762,"threads_started":1}
I20260808 02:49:47.124330 29172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.137.253@KRBTEST.COM; defaulting to no policy
I20260808 02:49:47.125560 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.125142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42426)
0808 02:49:47.125328 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.125448 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 02:49:47.125927 29105 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.253@KRBTEST.COM" created.
I20260808 02:49:47.127573 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.127135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42432)
0808 02:49:47.127342 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.127425 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:49:47.127797 29172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:35317: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab.
Entry for principal kudu/127.25.137.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab.
I20260808 02:49:47.166306 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.253:35317
--webserver_interface=127.25.137.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab
--principal=kudu/127.25.137.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:39393,127.25.137.253:35317,127.25.137.252:43275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:49:47.177175 29172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } attempt: 1
I20260808 02:49:47.178467 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.177964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42446)
0808 02:49:47.178180 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.178321 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 02:49:47.179502 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.179135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42450)
0808 02:49:47.179285 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.179364 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 02:49:47.179700 29172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:35317: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
I20260808 02:49:47.247951 29172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } attempt: 2
I20260808 02:49:47.249329 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.248694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42454)
0808 02:49:47.248975 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.249137 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 02:49:47.250427 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.249993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42462)
0808 02:49:47.250169 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.250259 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 02:49:47.250658 29172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:35317: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
I20260808 02:49:47.341957 29172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } attempt: 3
I20260808 02:49:47.343312 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.342697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42470)
0808 02:49:47.343019 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.343137 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260808 02:49:47.344406 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.343978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42472)
0808 02:49:47.344143 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.344231 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 02:49:47.344640 29172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:35317: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
I20260808 02:49:47.487893 29172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } attempt: 4
I20260808 02:49:47.489214 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.488571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42478)
0808 02:49:47.488897 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.489032 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
I20260808 02:49:47.490329 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.489906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42490)
0808 02:49:47.490083 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.490182 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260808 02:49:47.490578 29172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:35317: Network error: Client connection negotiation failed: client connection to 127.25.137.253:35317: connect: Connection refused (error 111)
W20260808 02:49:47.508579 29181 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:49:47.508922 29181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:47.509039 29181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:47.509151 29181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:47.516396 29181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:47.516485 29181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:47.516556 29181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:47.516626 29181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:47.516711 29181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:47.526706 29181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:39393,127.25.137.253:35317,127.25.137.252:43275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab
--principal=kudu/127.25.137.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.253:35317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.137.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:47.528012 29181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:47.529378 29181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:47.540622 29187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:47.540548 29190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:47.540586 29188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:47.541544 29181 server_base.cc:1034] running on GCE node
I20260808 02:49:47.542258 29181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:47.543581 29181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:47.544796 29181 hybrid_clock.cc:648] HybridClock initialized: now 1786157387544734 us; error 34 us; skew 500 ppm
I20260808 02:49:47.545145 29181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:47 dist-test-slave-k5rr krb5kdc[29052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157387, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:47.548254 29181 init.cc:377] Logged in from keytab as kudu/127.25.137.253@KRBTEST.COM (short username kudu)
I20260808 02:49:47.549731 29181 webserver.cc:460] Webserver started at http://127.25.137.253:41819/ using document root <none> and password file <none>
I20260808 02:49:47.550405 29181 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:47.550505 29181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:47.550805 29181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:47.553797 29181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/data/instance:
uuid: "917e2be839bc4d69821ae72609772221"
format_stamp: "Formatted at 2026-08-08 02:49:47 on dist-test-slave-k5rr"
server_key: "bb7ee212cca5b74fc2c62d35bb4ecafd"
server_key_iv: "f99b7c56b705d7995f14e65986c86382"
server_key_version: "encryptionkey@0"
I20260808 02:49:47.554584 29181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/wal/instance:
uuid: "917e2be839bc4d69821ae72609772221"
format_stamp: "Formatted at 2026-08-08 02:49:47 on dist-test-slave-k5rr"
server_key: "bb7ee212cca5b74fc2c62d35bb4ecafd"
server_key_iv: "f99b7c56b705d7995f14e65986c86382"
server_key_version: "encryptionkey@0"
I20260808 02:49:47.559495 29181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:49:47.562783 29199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:47.564384 29181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:49:47.564561 29181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/wal
uuid: "917e2be839bc4d69821ae72609772221"
format_stamp: "Formatted at 2026-08-08 02:49:47 on dist-test-slave-k5rr"
server_key: "bb7ee212cca5b74fc2c62d35bb4ecafd"
server_key_iv: "f99b7c56b705d7995f14e65986c86382"
server_key_version: "encryptionkey@0"
I20260808 02:49:47.564805 29181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:47.577865 29181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:47.582012 29181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:47.582233 29181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:47.598645 29181 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.253:35317
I20260808 02:49:47.598654 29287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.253:35317 every 8 connection(s)
I20260808 02:49:47.600024 29181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
I20260808 02:49:47.603614 29289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:47.609557 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 29181
I20260808 02:49:47.609745 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-1/wal/instance
I20260808 02:49:47.610287 26151 external_mini_cluster.cc:1428] Setting key 9154c838e68f9d65e8ec071f9164e0d7
I20260808 02:49:47.610184 29289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:49:47 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157387, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:47.632766 29175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.613274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57072 (local address 127.25.137.254:39393)
0808 02:49:47.613478 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:47.613484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:47.614687 (+ 1203us) server_negotiation.cc:407] Connection header received
0808 02:49:47.616937 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:47.616942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:47.617040 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:47.617127 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:47.618973 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.619840 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.621839 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.622260 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.622358 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:47.626310 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:47.626315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:47.626318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:47.626351 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:47.629505 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:47.630269 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:47.630274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:47.630277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:47.630455 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:47.630927 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:47.630933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:47.630935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:47.631309 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:47.631450 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:47.632174 (+ 724us) server_negotiation.cc:299] Negotiation successful
0808 02:49:47.632455 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:49:47.633546 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.613743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:39393 (local address 127.0.0.1:57072)
0808 02:49:47.614506 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.614550 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:47.616641 (+ 2091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:47.617390 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:47.617400 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:47.618054 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:47.618792 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:47.618816 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.619998 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.620004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:47.621692 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:47.621706 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.622474 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.622481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:47.622737 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:47.623417 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:47.623443 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:47.626093 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:47.629673 (+ 3580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:47.629680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:47.629693 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:47.630128 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:47.630565 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:47.630568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:47.630571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:47.630773 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:47.631431 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:47.631437 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:47.631771 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 02:49:47.632010 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 02:49:47.632416 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":298,"threads_started":1}
I20260808 02:49:47.635344 29289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:49:47 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157387, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.137.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.252@KRBTEST.COM" created.
I20260808 02:49:47.650445 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.636158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42496)
0808 02:49:47.636347 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.636365 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:47.636478 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:47.637733 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:47.637738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:47.638148 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:47.638320 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:47.638332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.639723 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.639728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:47.641033 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:47.641049 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.641858 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.641864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:47.642058 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:47.642627 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:47.642650 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:47.645099 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:47.648120 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:47.648126 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:47.648129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:47.648515 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:47.648904 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:47.648909 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:47.648912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:47.649114 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:47.649771 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:47.649774 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:47.649886 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 02:49:47.649985 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 02:49:47.650169 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:49:47.650800 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.636248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42496 (local address 127.25.137.253:35317)
0808 02:49:47.636950 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:47.636957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:47.636986 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:49:47.637064 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:47.637072 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:47.637296 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:47.637444 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:47.638491 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.639584 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.641158 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.641689 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.641811 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:47.645321 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:47.645332 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:47.645345 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:47.645373 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:47.647994 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:47.648645 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:47.648651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:47.648653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:47.648795 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:47.649220 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:47.649225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:47.649227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:47.649643 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:47.649834 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:47.650184 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 02:49:47.650454 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":386,"threads_started":1}
I20260808 02:49:47.653180 29289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:49:47.654196 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.653741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33778)
0808 02:49:47.653910 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.654027 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 02:49:47.654467 29203 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
I20260808 02:49:47.655984 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.655612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33790)
0808 02:49:47.655759 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.655862 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 02:49:47.656217 29289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:43275: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab.
Entry for principal kudu/127.25.137.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab.
I20260808 02:49:47.683828 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.252:43275
--webserver_interface=127.25.137.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab
--principal=kudu/127.25.137.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:39393,127.25.137.253:35317,127.25.137.252:43275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:49:47.705547 29289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } attempt: 1
I20260808 02:49:47.706926 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.706386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33800)
0808 02:49:47.706619 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.706750 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 02:49:47.708110 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.707700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33806)
0808 02:49:47.707898 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.707984 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 02:49:47.708333 29289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:43275: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
I20260808 02:49:47.776552 29289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } attempt: 2
I20260808 02:49:47.777661 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.777217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33812)
0808 02:49:47.777404 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.777499 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 02:49:47.778697 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.778369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33814)
0808 02:49:47.778515 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.778584 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 02:49:47.778939 29289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:43275: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
I20260808 02:49:47.789839 29172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } attempt: 5
Aug 08 02:49:47 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157387, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
I20260808 02:49:47.803529 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.790519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42498)
0808 02:49:47.790723 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.790799 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:47.790951 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:47.791320 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:47.791325 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:47.791760 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:47.791966 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:47.791979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.793093 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.793099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:47.794056 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:47.794071 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.794882 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.794889 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:47.795098 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:47.795686 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:47.795706 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:47.797841 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:47.800859 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:47.800865 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:47.800869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:47.801348 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:47.801744 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:47.801750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:47.801755 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:47.801905 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:47.802455 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:47.802460 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:47.802655 (+ 195us) client_negotiation.cc:769] Sending connection context
0808 02:49:47.802820 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 02:49:47.803105 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:49:47.803529 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.790588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42498 (local address 127.25.137.253:35317)
0808 02:49:47.790754 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:47.790761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:47.790878 (+ 117us) server_negotiation.cc:407] Connection header received
0808 02:49:47.791075 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:47.791082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:47.791169 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:47.791292 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:47.792130 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.792930 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.794226 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:47.794707 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:47.794844 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:47.798036 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:47.798041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:47.798044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:47.798079 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:47.800718 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:47.801473 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:47.801480 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:47.801482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:47.801617 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:47.802016 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:47.802022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:47.802024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:47.802329 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:47.802475 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:47.802947 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 02:49:47.803165 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:49:47.804636 29172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:49:47.805819 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.805369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33824)
0808 02:49:47.805525 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.805604 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 02:49:47.806749 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.806418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33826)
0808 02:49:47.806569 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.806641 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 02:49:47.806947 29172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:43275: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
I20260808 02:49:47.858186 29172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } attempt: 1
I20260808 02:49:47.859386 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.858890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33840)
0808 02:49:47.859104 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.859200 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 02:49:47.860388 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.860038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33852)
0808 02:49:47.860192 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.860267 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 02:49:47.860641 29172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:43275: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
I20260808 02:49:47.870126 29289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } attempt: 3
I20260808 02:49:47.871101 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.870633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33858)
0808 02:49:47.870845 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.870946 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 02:49:47.872133 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.871740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33870)
0808 02:49:47.871947 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.872021 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260808 02:49:47.872354 29289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:43275: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
I20260808 02:49:47.928870 29172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } attempt: 2
I20260808 02:49:47.930030 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.929526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33874)
0808 02:49:47.929766 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.929861 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:49:47.931006 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:47.930669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33880)
0808 02:49:47.930822 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:47.930895 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 02:49:47.931200 29172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:43275: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
I20260808 02:49:48.015570 29289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } attempt: 4
I20260808 02:49:48.016887 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.016404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33894)
0808 02:49:48.016627 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:48.016719 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 02:49:48.017874 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.017535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33910)
0808 02:49:48.017688 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:48.017758 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 02:49:48.018069 29289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:43275: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
W20260808 02:49:48.023524 29303 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:49:48.023835 29303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:48.023929 29303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:48.024039 29303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:48.031307 29303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:48.031395 29303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:48.031507 29303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:48.031574 29303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:48.031656 29303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:48.037390 29172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } attempt: 3
I20260808 02:49:48.038439 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.037976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33920)
0808 02:49:48.038162 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:48.038249 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 02:49:48.039386 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.039022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33924)
0808 02:49:48.039186 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:48.039254 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 02:49:48.039590 29172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:43275: Network error: Client connection negotiation failed: client connection to 127.25.137.252:43275: connect: Connection refused (error 111)
I20260808 02:49:48.041672 29303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:39393,127.25.137.253:35317,127.25.137.252:43275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab
--principal=kudu/127.25.137.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.252:43275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.137.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:48.042886 29303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:48.044198 29303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:48.055361 29308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:48.055482 29313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:48.055493 29309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:48.056491 29303 server_base.cc:1034] running on GCE node
I20260808 02:49:48.057231 29303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:48.058511 29303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:48.059723 29303 hybrid_clock.cc:648] HybridClock initialized: now 1786157388059668 us; error 35 us; skew 500 ppm
I20260808 02:49:48.060041 29303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:48 dist-test-slave-k5rr krb5kdc[29052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157388, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:48.063300 29303 init.cc:377] Logged in from keytab as kudu/127.25.137.252@KRBTEST.COM (short username kudu)
I20260808 02:49:48.064711 29303 webserver.cc:460] Webserver started at http://127.25.137.252:42591/ using document root <none> and password file <none>
I20260808 02:49:48.065466 29303 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:48.065572 29303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:48.065896 29303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:48.068848 29303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/data/instance:
uuid: "9eaa6381375a40dcba47397273ff0285"
format_stamp: "Formatted at 2026-08-08 02:49:48 on dist-test-slave-k5rr"
server_key: "89abdff184c071e6738397674f46d502"
server_key_iv: "9a8a470adf35f8ad7fe3d2a488903d63"
server_key_version: "encryptionkey@0"
I20260808 02:49:48.069772 29303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/wal/instance:
uuid: "9eaa6381375a40dcba47397273ff0285"
format_stamp: "Formatted at 2026-08-08 02:49:48 on dist-test-slave-k5rr"
server_key: "89abdff184c071e6738397674f46d502"
server_key_iv: "9a8a470adf35f8ad7fe3d2a488903d63"
server_key_version: "encryptionkey@0"
I20260808 02:49:48.074795 29303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:49:48.078205 29322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:48.079512 29303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:49:48.079659 29303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/wal
uuid: "9eaa6381375a40dcba47397273ff0285"
format_stamp: "Formatted at 2026-08-08 02:49:48 on dist-test-slave-k5rr"
server_key: "89abdff184c071e6738397674f46d502"
server_key_iv: "9a8a470adf35f8ad7fe3d2a488903d63"
server_key_version: "encryptionkey@0"
I20260808 02:49:48.079860 29303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:48.094065 29303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:48.098007 29303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:48.098246 29303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:48.116169 29303 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.252:43275
I20260808 02:49:48.116178 29396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.252:43275 every 8 connection(s)
I20260808 02:49:48.117590 29303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
I20260808 02:49:48.121295 29398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:48.126914 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 29303
I20260808 02:49:48.127089 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157019169307-26151-0/minicluster-data/master-2/wal/instance
I20260808 02:49:48.127550 26151 external_mini_cluster.cc:1428] Setting key a381f5dbaeea5bcc59a9bd4d656cff28
I20260808 02:49:48.127871 29398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:49:48 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157388, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
Aug 08 02:49:48 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
I20260808 02:49:48.147217 29175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.130620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57088 (local address 127.25.137.254:39393)
0808 02:49:48.130791 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.130854 (+ 63us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.131354 (+ 500us) server_negotiation.cc:407] Connection header received
0808 02:49:48.132546 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.132552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.132621 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.132706 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.133989 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.134684 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.136199 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.136855 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.136948 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.140593 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:48.140597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:48.140601 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:48.140632 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:48.143730 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.144543 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.144549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.144552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.144722 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.145256 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.145259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.145262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.145557 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:48.145719 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.146610 (+ 891us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.146860 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:49:48.148187 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.129415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33932 (local address 127.25.137.252:43275)
0808 02:49:48.130163 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.130176 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.130222 (+ 46us) server_negotiation.cc:407] Connection header received
0808 02:49:48.132280 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.132304 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.132527 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.132702 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.134189 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.135333 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.136780 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.137618 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.137747 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.141367 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:48.141378 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:48.141394 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:48.141431 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:48.144606 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.145518 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.145526 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.145529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.145678 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.146090 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.146093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.146096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.146447 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:48.146746 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.147115 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.147363 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":535,"thread_start_us":319,"threads_started":1}
I20260808 02:49:48.148234 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.130566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:39393 (local address 127.0.0.1:57088)
0808 02:49:48.131201 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:48.131241 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:48.132286 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:48.132792 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:48.132799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:48.133475 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:48.133824 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:48.133841 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.134840 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.134843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:48.136045 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:48.136061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.137088 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.137096 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:48.137451 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.138213 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:48.138234 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:48.140414 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:48.143865 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:48.143874 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:48.143882 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:48.144419 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:48.144858 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:48.144864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:48.144868 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:48.145138 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:48.145716 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:48.145722 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:48.146167 (+ 445us) client_negotiation.cc:769] Sending connection context
0808 02:49:48.146462 (+ 295us) client_negotiation.cc:240] Negotiation successful
0808 02:49:48.146932 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":36,"thread_start_us":286,"threads_started":1}
I20260808 02:49:48.150565 29398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:49:48 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157388, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
Aug 08 02:49:48 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:48.163739 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.151560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42510 (local address 127.25.137.253:35317)
0808 02:49:48.151750 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.151756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.151774 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:49:48.151834 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.151838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.151895 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.151976 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.152662 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.153381 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.154512 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.154931 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.155036 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.158259 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:48.158263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:48.158267 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:48.158302 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:48.161093 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.161884 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.161888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.161891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.162018 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.162451 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.162456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.162460 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.162733 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:48.162857 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.163320 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.163480 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:49:48.164124 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.151345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:35317 (local address 127.0.0.1:42510)
0808 02:49:48.151578 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:48.151593 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:48.151685 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:48.152026 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:48.152030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:48.152355 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:48.152527 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:48.152538 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.153518 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.153524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:48.154381 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:48.154392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.155062 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.155067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:48.155262 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.155733 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:48.155749 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:48.158071 (+ 2322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:48.161220 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:48.161226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:48.161231 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:48.161746 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:48.162123 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:48.162128 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:48.162132 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:48.162327 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:48.162865 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:48.162870 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:48.163040 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 02:49:48.163229 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 02:49:48.163531 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260808 02:49:48.165179 29398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:49:48.166724 29175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.151298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57096 (local address 127.25.137.254:39393)
0808 02:49:48.151477 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.151495 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.151828 (+ 333us) server_negotiation.cc:407] Connection header received
0808 02:49:48.151972 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.151978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.152072 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.152166 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.153482 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.154226 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.155427 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.155854 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.155983 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.160680 (+ 4697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:48.160685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:48.160688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:48.160720 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:48.164072 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.164782 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.164787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.164790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.164977 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.165367 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.165372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.165376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.165651 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:48.165981 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.166228 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.166437 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 02:49:48 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157388, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
Aug 08 02:49:48 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
I20260808 02:49:48.178090 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.165798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33938)
0808 02:49:48.165938 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:48.165955 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:48.166062 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:48.166464 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:48.166468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:48.166861 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:48.167015 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:48.167025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.168027 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.168032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:48.169066 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:48.169080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.170029 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.170033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:48.170212 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.170716 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:48.170733 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:48.172827 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:48.175809 (+ 2982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:48.175814 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:48.175818 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:48.176148 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:48.176523 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:48.176527 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:48.176529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:48.176674 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:48.177336 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:48.177340 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:48.177473 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 02:49:48.177588 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 02:49:48.177794 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 02:49:48.178321 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.166118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.25.137.252:43275)
0808 02:49:48.166250 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.166254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.166265 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:49:48.166306 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.166309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.166354 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.166428 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.167139 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.167867 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.169186 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.169879 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.170012 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.173024 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:48.173031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:48.173036 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:48.173071 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:48.175681 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.176261 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.176266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.176269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.176410 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.176789 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.176793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.176797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.177196 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:48.177350 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.177709 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.177945 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":56}
I20260808 02:49:48.182698 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.168299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.25.137.253:35317)
0808 02:49:48.168481 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.168487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.168550 (+ 63us) server_negotiation.cc:407] Connection header received
0808 02:49:48.168713 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.168718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.168783 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.168869 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.170344 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.171035 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.172546 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.172966 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.173091 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.177423 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:48.177428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:48.177431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:48.177461 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:48.180284 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.180993 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.180996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.180998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.181169 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.181541 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.181544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.181546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.181781 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:48.181990 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.182261 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.182429 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:49:48.183912 29398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285: Bootstrap starting.
I20260808 02:49:48.187321 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:48.187341 29398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:48.188321 29398 log.cc:824] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:48.191138 29398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285: No bootstrap required, opened a new log
I20260808 02:49:48.190151 29133 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3a199450f9c147e1a1f5ccc24582b356"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57096
I20260808 02:49:48.198230 29398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } }
I20260808 02:49:48.198576 29398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eaa6381375a40dcba47397273ff0285, State: Initialized, Role: FOLLOWER
I20260808 02:49:48.199004 29398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } }
I20260808 02:49:48.199918 29410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } } }
I20260808 02:49:48.200344 29410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:48.200896 29398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:49:48.209839 29426 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:49:48.209968 29426 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:49:48.216802 29172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } attempt: 4
I20260808 02:49:48.207625 29303 master_runner.cc:186] Error getting master registration for 127.25.137.254:39393: OK, instance_id { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" instance_seqno: 1786157387076903 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:49:48.218654 29303 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 02:49:48 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157387, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
I20260808 02:49:48.234043 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.217563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33950)
0808 02:49:48.217852 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:48.217872 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:48.217998 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:48.218390 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:48.218395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:48.218924 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:48.219136 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:48.219149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.220561 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.220566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:48.222074 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:48.222090 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.223117 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.223123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:48.223337 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.224008 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:48.224031 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:48.226793 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:48.230868 (+ 4075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:48.230873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:48.230878 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:48.231429 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:48.231973 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:48.231978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:48.232000 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:48.232177 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:48.232940 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:48.232945 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:48.233161 (+ 216us) client_negotiation.cc:769] Sending connection context
0808 02:49:48.233339 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 02:49:48.233627 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260808 02:49:48.234047 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.217810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33950 (local address 127.25.137.252:43275)
0808 02:49:48.218023 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.218029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.218054 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:49:48.218119 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.218128 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.218211 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.218339 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.219294 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.220423 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.222228 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.222978 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.223114 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.226989 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:48.226997 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:48.227002 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:48.227040 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:48.230710 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.231551 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.231557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.231561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.231719 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.232301 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.232306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.232310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.232807 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:48.232948 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.233442 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.233671 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 02:49:48.240424 29172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356: Bootstrap starting.
I20260808 02:49:48.244521 29172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:48.245862 29172 log.cc:824] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:48.249383 29172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356: No bootstrap required, opened a new log
I20260808 02:49:48.256922 29172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } }
I20260808 02:49:48.257373 29172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a199450f9c147e1a1f5ccc24582b356, State: Initialized, Role: FOLLOWER
I20260808 02:49:48.257882 29172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } }
I20260808 02:49:48.258930 29432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } } }
I20260808 02:49:48.259347 29432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:48.260125 29172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:48.260317 29133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 02:49:48.260521 29133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:48.263125 29133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } }
I20260808 02:49:48.264436 29133 leader_election.cc:288] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [CANDIDATE]: Term 1 election: Requested vote from peers 917e2be839bc4d69821ae72609772221 (127.25.137.253:35317), 9eaa6381375a40dcba47397273ff0285 (127.25.137.252:43275)
I20260808 02:49:48.266390 29356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3a199450f9c147e1a1f5ccc24582b356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9eaa6381375a40dcba47397273ff0285"
I20260808 02:49:48.266991 29356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:48.271191 29356 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a199450f9c147e1a1f5ccc24582b356 in term 1.
I20260808 02:49:48.272300 29105 leader_election.cc:302] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a199450f9c147e1a1f5ccc24582b356, 9eaa6381375a40dcba47397273ff0285; no voters:
I20260808 02:49:48.272924 29432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:48.284899 29432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [term 1 LEADER]: Becoming Leader. State: Replica: 3a199450f9c147e1a1f5ccc24582b356, State: Running, Role: LEADER
I20260808 02:49:48.286489 29432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } }
I20260808 02:49:48.303720 29438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a199450f9c147e1a1f5ccc24582b356. Latest consensus state: current_term: 1 leader_uuid: "3a199450f9c147e1a1f5ccc24582b356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } } }
I20260808 02:49:48.304344 29438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:48.315357 29448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260808 02:49:48.316792 29108 tablet.cc:2369] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:49:48.317701 29289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } attempt: 5
W20260808 02:49:48.351168 29332 tablet.cc:2369] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:49:48.354657 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.307484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.25.137.253:35317)
0808 02:49:48.307714 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.307722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.317326 (+ 9604us) server_negotiation.cc:407] Connection header received
0808 02:49:48.326636 (+ 9310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.326643 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.326740 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.326844 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.331215 (+ 4371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.332371 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.334869 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.335567 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.335671 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.340276 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:48.340283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:48.340288 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:48.340344 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:48.343978 (+ 3634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.346314 (+ 2336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.346320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.346324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.346506 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.348825 (+ 2319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.348831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.348835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.349536 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:48.349798 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.351086 (+ 1288us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.354225 (+ 3139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1060,"server-negotiator.queue_time_us":88}
I20260808 02:49:48.355118 29448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:49:48.359593 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.317929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33952 (local address 127.25.137.252:43275)
0808 02:49:48.318106 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.318113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.337250 (+ 19137us) server_negotiation.cc:407] Connection header received
0808 02:49:48.337497 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.337502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.337599 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.337710 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.340707 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.341796 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.343738 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.344455 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.344557 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.348528 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:48.348535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:48.348540 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:48.348579 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:48.353847 (+ 5268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.356260 (+ 2413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.356267 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.356271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.356490 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.357505 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.357512 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.357516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.357914 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:48.358187 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.358858 (+ 671us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.359127 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 02:49:48 dist-test-slave-k5rr krb5kdc[29052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157387, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
I20260808 02:49:48.375882 29175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.311387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57100 (local address 127.25.137.254:39393)
0808 02:49:48.311626 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.311633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.330409 (+ 18776us) server_negotiation.cc:407] Connection header received
0808 02:49:48.330498 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.330505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.330598 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.330713 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.337923 (+ 7210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.338996 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.349139 (+ 10143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.349863 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.349970 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.364588 (+ 14618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:48.364595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:48.364600 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:48.364644 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:48.368244 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.372226 (+ 3982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.372232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.372235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.372428 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.373926 (+ 1498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.373932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.373935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.374278 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:48.374558 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.375176 (+ 618us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.375453 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 02:49:48.379455 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.318454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:43275 (local address 127.0.0.1:33960)
0808 02:49:48.318868 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:48.318889 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:48.319016 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:48.341042 (+ 22026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:48.341048 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:48.341603 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:48.341853 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:48.341867 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.350890 (+ 9023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.350897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:48.352333 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:48.352351 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.358880 (+ 6529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.358886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:48.359104 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.359665 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:48.359689 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:48.362339 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:48.371231 (+ 8892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:48.371236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:48.371241 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:48.371781 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:48.373032 (+ 1251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:48.373038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:48.373042 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:48.373246 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:48.378430 (+ 5184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:48.378436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:48.378600 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 02:49:48.378728 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 02:49:48.379050 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281}
I20260808 02:49:48.381481 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.319259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33960 (local address 127.25.137.252:43275)
0808 02:49:48.337304 (+ 18045us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.337311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.337340 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:49:48.337407 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.337413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.337503 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.342013 (+ 4510us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.342327 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.343613 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.355781 (+ 12168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.356518 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.362507 (+ 5989us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.362734 (+ 227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:48.362740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:48.362745 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:48.362774 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:48.371077 (+ 8303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.371954 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.371960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.371963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.372877 (+ 914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:48.373395 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:48.373400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:48.373403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:48.373760 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:48.379829 (+ 6069us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.380772 (+ 943us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.381051 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17891,"thread_start_us":18333,"threads_started":1}
I20260808 02:49:48.367719 29076 master_runner.cc:186] Error getting master registration for 127.25.137.253:35317: OK, instance_id { permanent_uuid: "917e2be839bc4d69821ae72609772221" instance_seqno: 1786157387593983 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:49:48.383387 29076 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:49:48.386965 29356 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [term 1 FOLLOWER]: Refusing update from remote peer 3a199450f9c147e1a1f5ccc24582b356: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:49:48.387929 29438 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:49:48.388063 29289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221: Bootstrap starting.
I20260808 02:49:48.397342 29289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:48.399940 29289 log.cc:824] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:48.412887 29410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a199450f9c147e1a1f5ccc24582b356. Latest consensus state: current_term: 1 leader_uuid: "3a199450f9c147e1a1f5ccc24582b356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } } }
I20260808 02:49:48.413205 29410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:48.415040 29289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221: No bootstrap required, opened a new log
I20260808 02:49:48.427695 29410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a199450f9c147e1a1f5ccc24582b356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } } }
I20260808 02:49:48.427968 29410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:48.433524 29448 catalog_manager.cc:1347] Generated new cluster ID: 8b9a1c71b3f5444eb4ee93f8f0b6e1ca
I20260808 02:49:48.433650 29448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:48.434131 29438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a199450f9c147e1a1f5ccc24582b356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } } }
I20260808 02:49:48.434445 29438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:48.436754 29289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } }
I20260808 02:49:48.437268 29289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917e2be839bc4d69821ae72609772221, State: Initialized, Role: FOLLOWER
I20260808 02:49:48.437811 29289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } }
I20260808 02:49:48.439075 29473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } } }
I20260808 02:49:48.439801 29473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:48.440207 29289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:48.440613 29239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:48.440232 29240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3a199450f9c147e1a1f5ccc24582b356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "917e2be839bc4d69821ae72609772221"
I20260808 02:49:48.448448 29239 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [term 1 FOLLOWER]: Refusing update from remote peer 3a199450f9c147e1a1f5ccc24582b356: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:49:48.452015 29240 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a199450f9c147e1a1f5ccc24582b356 in term 1.
I20260808 02:49:48.456228 29432 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [LEADER]: Connected to new peer: Peer: permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:49:48.458606 29448 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:48.461913 29448 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:48.473754 29448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356: Generated new TSK 0
I20260808 02:49:48.474695 29448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:49:48.501935 29473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a199450f9c147e1a1f5ccc24582b356. Latest consensus state: current_term: 1 leader_uuid: "3a199450f9c147e1a1f5ccc24582b356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } } }
I20260808 02:49:48.501902 29490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a199450f9c147e1a1f5ccc24582b356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } } }
I20260808 02:49:48.502332 29490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:48.502393 29438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a199450f9c147e1a1f5ccc24582b356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a199450f9c147e1a1f5ccc24582b356" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 39393 } } peers { permanent_uuid: "917e2be839bc4d69821ae72609772221" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 35317 } } peers { permanent_uuid: "9eaa6381375a40dcba47397273ff0285" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 43275 } } }
I20260808 02:49:48.502329 29473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:48.502727 29438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a199450f9c147e1a1f5ccc24582b356 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:48.532918 29504 catalog_manager.cc:1259] Loaded cluster ID: 8b9a1c71b3f5444eb4ee93f8f0b6e1ca
I20260808 02:49:48.533046 29504 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221: loading cluster ID for follower catalog manager: success
I20260808 02:49:48.536772 29504 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221: acquiring CA information for follower catalog manager: success
I20260808 02:49:48.539152 29504 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 917e2be839bc4d69821ae72609772221: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 02:49:48.599560 26151 mini_kdc.cc:348] Time spent kdestroy: real 0.114s user 0.000s sys 0.029s
W20260808 02:49:48.663757 29175 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:48.652782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57112 (local address 127.25.137.254:39393)
0808 02:49:48.652994 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.653001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.661628 (+ 8627us) server_negotiation.cc:407] Connection header received
0808 02:49:48.661984 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.661991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.662101 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.662185 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.663505 (+ 1320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57112: BlockingRecv error: recv got EOF from 127.0.0.1:57112 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 02:49:48.667088 29296 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:48.657431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42542 (local address 127.25.137.253:35317)
0808 02:49:48.657608 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.657614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.664904 (+ 7290us) server_negotiation.cc:407] Connection header received
0808 02:49:48.665433 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.665439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.665535 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.665617 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.666833 (+ 1216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42542: BlockingRecv error: recv got EOF from 127.0.0.1:42542 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 02:49:48.667528 29459 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:48.665094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33962 (local address 127.25.137.252:43275)
0808 02:49:48.665324 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.665329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.665953 (+ 624us) server_negotiation.cc:407] Connection header received
0808 02:49:48.666203 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.666208 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.666317 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.666388 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.667297 (+ 909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33962: BlockingRecv error: recv got EOF from 127.0.0.1:33962 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:49:48.729612 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.715363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.25.137.253:35317)
0808 02:49:48.715555 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.715561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.721647 (+ 6086us) server_negotiation.cc:407] Connection header received
0808 02:49:48.721979 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.721986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.722216 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.722380 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:48.723168 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.724314 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.726468 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.727146 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.727252 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.728050 (+ 798us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:48.728794 (+ 744us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:48.728913 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.729191 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.729291 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 02:49:48.773068 29459 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:48.763137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33976 (local address 127.25.137.252:43275)
0808 02:49:48.763318 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.763324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.770781 (+ 7457us) server_negotiation.cc:407] Connection header received
0808 02:49:48.771330 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.771337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.771469 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.771567 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.772712 (+ 1145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33976: BlockingRecv error: recv got EOF from 127.0.0.1:33976 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:49:48.818425 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.804274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42562 (local address 127.25.137.253:35317)
0808 02:49:48.804501 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.804507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.811845 (+ 7338us) server_negotiation.cc:407] Connection header received
0808 02:49:48.812367 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.812374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.812459 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.812523 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:48.813176 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.814156 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.816092 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.816855 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.816946 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.817564 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:48.817728 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:48.817788 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.818006 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.818089 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 02:49:48.834865 29459 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:48.829963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.25.137.252:43275)
0808 02:49:48.830195 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.830202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.833565 (+ 3363us) server_negotiation.cc:407] Connection header received
0808 02:49:48.833762 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.833767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.833844 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.833923 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.834685 (+ 762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33988: BlockingRecv error: recv got EOF from 127.0.0.1:33988 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:49:48.846808 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.840877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.25.137.253:35317)
0808 02:49:48.841074 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.841079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.841564 (+ 485us) server_negotiation.cc:407] Connection header received
0808 02:49:48.841737 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.841743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.841829 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.841908 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:48.842295 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.843266 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.845038 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.845530 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.845643 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.846069 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:48.846173 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:48.846246 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.846393 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.846475 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 02:49:48.854595 29459 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:48.852403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34004 (local address 127.25.137.252:43275)
0808 02:49:48.852593 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.852600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.853160 (+ 560us) server_negotiation.cc:407] Connection header received
0808 02:49:48.853445 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.853451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.853552 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.853613 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.854425 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34004: BlockingRecv error: recv got EOF from 127.0.0.1:34004 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:49:48.867393 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.862049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42588 (local address 127.25.137.253:35317)
0808 02:49:48.862318 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.862325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.862791 (+ 466us) server_negotiation.cc:407] Connection header received
0808 02:49:48.862960 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.862966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.863038 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.863115 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:48.863518 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.864337 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.865741 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.866186 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.866326 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.866695 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:48.866794 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:48.866868 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.867002 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.867080 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
W20260808 02:49:48.878857 29459 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:48.873460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34006 (local address 127.25.137.252:43275)
0808 02:49:48.873701 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.873707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.877491 (+ 3784us) server_negotiation.cc:407] Connection header received
0808 02:49:48.877676 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.877682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.877771 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.877857 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.878604 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34006: BlockingRecv error: recv got EOF from 127.0.0.1:34006 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
I20260808 02:49:48.900534 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.894909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.25.137.253:35317)
0808 02:49:48.895125 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.895131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.895641 (+ 510us) server_negotiation.cc:407] Connection header received
0808 02:49:48.895850 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.895855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.895930 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.896010 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:48.896459 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.897415 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.898735 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.899199 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.899311 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.899768 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:48.899885 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:48.899971 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.900118 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.900200 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 02:49:48.908885 29459 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:48.906892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.25.137.252:43275)
0808 02:49:48.907095 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.907102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.907583 (+ 481us) server_negotiation.cc:407] Connection header received
0808 02:49:48.907741 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.907745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.907828 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.907909 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.908641 (+ 732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34016: BlockingRecv error: recv got EOF from 127.0.0.1:34016 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:49:48.934195 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.928069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42604 (local address 127.25.137.253:35317)
0808 02:49:48.928318 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.928327 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.928885 (+ 558us) server_negotiation.cc:407] Connection header received
0808 02:49:48.929134 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.929142 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.929221 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.929314 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:48.929833 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.930702 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.932149 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.932668 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.932822 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.933364 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:48.933482 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:48.933576 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.933762 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.933864 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 02:49:48.942651 29459 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:48.940455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34020 (local address 127.25.137.252:43275)
0808 02:49:48.940658 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.940666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.941194 (+ 528us) server_negotiation.cc:407] Connection header received
0808 02:49:48.941383 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.941389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.941474 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.941571 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.942405 (+ 834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34020: BlockingRecv error: recv got EOF from 127.0.0.1:34020 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:49:48.983199 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:48.977742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42612 (local address 127.25.137.253:35317)
0808 02:49:48.977994 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.978001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.978505 (+ 504us) server_negotiation.cc:407] Connection header received
0808 02:49:48.978733 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.978738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.978809 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.978889 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:48.979324 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.980069 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.981446 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:48.981948 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:48.982055 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:48.982437 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:48.982545 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:48.982634 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:48.982755 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 02:49:48.982835 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260808 02:49:48.993919 29459 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:48.992096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.25.137.252:43275)
0808 02:49:48.992322 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:48.992328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:48.992553 (+ 225us) server_negotiation.cc:407] Connection header received
0808 02:49:48.992696 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:48.992702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:48.992800 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:48.992922 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:48.993677 (+ 755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34028: BlockingRecv error: recv got EOF from 127.0.0.1:34028 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:49:49.066141 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:49.061356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.25.137.253:35317)
0808 02:49:49.061543 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:49.061547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:49.061920 (+ 373us) server_negotiation.cc:407] Connection header received
0808 02:49:49.062087 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:49.062091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:49.062161 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:49.062245 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:49.062577 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.063323 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.064614 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.065059 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.065150 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:49.065535 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:49.065636 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:49.065701 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:49.065838 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 02:49:49.065932 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 02:49:49.073738 29459 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:49.071801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34042 (local address 127.25.137.252:43275)
0808 02:49:49.071976 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:49.071983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:49.072365 (+ 382us) server_negotiation.cc:407] Connection header received
0808 02:49:49.072557 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:49.072561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:49.072644 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:49.072727 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:49.073568 (+ 841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34042: BlockingRecv error: recv got EOF from 127.0.0.1:34042 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:49:49.210253 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:49.205113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42636 (local address 127.25.137.253:35317)
0808 02:49:49.205325 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:49.205333 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:49.205673 (+ 340us) server_negotiation.cc:407] Connection header received
0808 02:49:49.205894 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:49.205899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:49.205971 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:49.206050 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:49.206490 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.207200 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.208573 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.208982 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.209119 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:49.209527 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:49.209638 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:49.209716 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:49.209863 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 02:49:49.209944 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:49:49.212122 29426 catalog_manager.cc:1259] Loaded cluster ID: 8b9a1c71b3f5444eb4ee93f8f0b6e1ca
I20260808 02:49:49.212220 29426 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285: loading cluster ID for follower catalog manager: success
I20260808 02:49:49.215794 29426 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285: acquiring CA information for follower catalog manager: success
I20260808 02:49:49.217762 29426 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9eaa6381375a40dcba47397273ff0285: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 02:49:49.218386 29459 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:49.216206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34050 (local address 127.25.137.252:43275)
0808 02:49:49.216452 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:49.216460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:49.217044 (+ 584us) server_negotiation.cc:407] Connection header received
0808 02:49:49.217228 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:49.217232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:49.217328 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:49.217414 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:49.218198 (+ 784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34050: BlockingRecv error: recv got EOF from 127.0.0.1:34050 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:49:49.484911 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:49.478615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42650 (local address 127.25.137.253:35317)
0808 02:49:49.478836 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:49.478843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:49.479044 (+ 201us) server_negotiation.cc:407] Connection header received
0808 02:49:49.479232 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:49.479239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:49.479346 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:49.479463 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:49.479997 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.481070 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.482851 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.483519 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.483646 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:49.484123 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:49.484228 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:49.484321 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:49.484525 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 02:49:49.484616 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:49:49.496524 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:49.491002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34056 (local address 127.25.137.252:43275)
0808 02:49:49.491196 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:49.491202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:49.491467 (+ 265us) server_negotiation.cc:407] Connection header received
0808 02:49:49.491665 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:49.491670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:49.491827 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:49.491925 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:49.492311 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.492985 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.494378 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.494786 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.494894 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:49.495431 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:49.495960 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:49.496094 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:49.496195 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 02:49:49.496286 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:49:49.511734 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:49.502657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.25.137.253:35317)
0808 02:49:49.502852 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:49.502860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:49.505911 (+ 3051us) server_negotiation.cc:407] Connection header received
0808 02:49:49.506181 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:49.506188 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:49.506278 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:49.506351 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:49.507067 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.508058 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.509924 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.510553 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.510671 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:49.511040 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:49.511138 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:49.511212 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:49.511349 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 02:49:49.511429 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:49:49.512609 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:49.505908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.25.137.252:43275)
0808 02:49:49.506083 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:49.506089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:49.506882 (+ 793us) server_negotiation.cc:407] Connection header received
0808 02:49:49.507080 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:49.507085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:49.507167 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:49.507224 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:49.507914 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.508792 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.510532 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.511150 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.511261 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:49.511814 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:49.511967 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:49.512059 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:49.512241 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 02:49:49.512329 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:49:49.512897 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:49.502555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57126 (local address 127.25.137.254:39393)
0808 02:49:49.506486 (+ 3931us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:49.506491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:49.506510 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:49:49.506627 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:49.506633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:49.506838 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:49.507009 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:49.507493 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.508408 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.510350 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:49.510930 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:49.511049 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:49.511758 (+ 709us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:49.512219 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:49.512326 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:49.512520 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 02:49:49.512603 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3800,"thread_start_us":3687,"threads_started":1}
I20260808 02:49:49.518028 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29076
I20260808 02:49:49.519066 29166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:49.632771 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 29076
W20260808 02:49:49.655936 29203 connection.cc:535] server connection from 127.0.0.1:42498 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 02:49:49.655969 29326 connection.cc:535] server connection from 127.0.0.1:33950 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:49:49.656651 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29181
I20260808 02:49:49.657809 29275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:49.765206 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 29181
I20260808 02:49:49.791031 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29303
I20260808 02:49:49.792136 29387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:49.899111 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 29303
2026-08-08T02:49:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4821 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:49:50 dist-test-slave-k5rr krb5kdc[29689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:49:50 dist-test-slave-k5rr krb5kdc[29689](info): set up 2 sockets
Aug 08 02:49:50 dist-test-slave-k5rr krb5kdc[29689](info): commencing operation
krb5kdc: starting...
W20260808 02:49:51.142813 26151 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/test-user.keytab.
Aug 08 02:49:51 dist-test-slave-k5rr krb5kdc[29689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:49:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:49:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
Entry for principal kudu/127.25.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab.
I20260808 02:49:51.484200 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:34831
--webserver_interface=127.25.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:34831,127.25.137.253:43149,127.25.137.252:37675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:49:51.834633 29716 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:49:51.834977 29716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:51.835059 29716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:51.835150 29716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:51.842453 29716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:51.842557 29716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:51.842633 29716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:51.842691 29716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:51.842770 29716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:51.852712 29716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:34831,127.25.137.253:43149,127.25.137.252:37675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.254.keytab
--principal=kudu/127.25.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.254:34831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:51.854043 29716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:51.855428 29716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:51.867394 29727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:51.867323 29723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:51.867440 29721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:51.868386 29716 server_base.cc:1034] running on GCE node
I20260808 02:49:51.869236 29716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:51.870599 29716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:51.871840 29716 hybrid_clock.cc:648] HybridClock initialized: now 1786157391871738 us; error 48 us; skew 500 ppm
I20260808 02:49:51.872165 29716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:51 dist-test-slave-k5rr krb5kdc[29689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157391, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:51.875239 29716 init.cc:377] Logged in from keytab as kudu/127.25.137.254@KRBTEST.COM (short username kudu)
I20260808 02:49:51.876672 29716 webserver.cc:460] Webserver started at http://127.25.137.254:33669/ using document root <none> and password file <none>
I20260808 02:49:51.877401 29716 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:51.877506 29716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:51.877828 29716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:51.880707 29716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/data/instance:
uuid: "60d20d409c8346f784c105f546c190b6"
format_stamp: "Formatted at 2026-08-08 02:49:51 on dist-test-slave-k5rr"
server_key: "ebfc73015c0ba623ebccd760aede31c6"
server_key_iv: "7d3a084d85ba1c037365890525335edd"
server_key_version: "encryptionkey@0"
I20260808 02:49:51.881542 29716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/wal/instance:
uuid: "60d20d409c8346f784c105f546c190b6"
format_stamp: "Formatted at 2026-08-08 02:49:51 on dist-test-slave-k5rr"
server_key: "ebfc73015c0ba623ebccd760aede31c6"
server_key_iv: "7d3a084d85ba1c037365890525335edd"
server_key_version: "encryptionkey@0"
I20260808 02:49:51.886493 29716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:49:51.889916 29742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:51.891194 29716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:49:51.891350 29716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/wal
uuid: "60d20d409c8346f784c105f546c190b6"
format_stamp: "Formatted at 2026-08-08 02:49:51 on dist-test-slave-k5rr"
server_key: "ebfc73015c0ba623ebccd760aede31c6"
server_key_iv: "7d3a084d85ba1c037365890525335edd"
server_key_version: "encryptionkey@0"
I20260808 02:49:51.891553 29716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:51.925036 29716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:51.929025 29716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:51.929284 29716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:51.946942 29716 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.254:34831
I20260808 02:49:51.946964 29814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.254:34831 every 8 connection(s)
I20260808 02:49:51.948235 29716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/data/info.pb
I20260808 02:49:51.951879 29815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:51.958148 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 29716
I20260808 02:49:51.958343 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-0/wal/instance
I20260808 02:49:51.958922 26151 external_mini_cluster.cc:1428] Setting key c1d6592b76218c09c1e6fd4a84f41bec
I20260808 02:49:51.958618 29815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:49:51 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157391, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:51.980690 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:51.963259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42190 (local address 127.25.137.254:34831)
0808 02:49:51.963916 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:51.963923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:51.963972 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:49:51.965229 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:51.965236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:51.965549 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:51.965749 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:51.966869 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:51.968447 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:51.969876 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:51.970646 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:51.970809 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:51.973996 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:51.974005 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:51.974018 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:51.974053 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:51.976764 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:51.977628 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:51.977635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:51.977637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:51.977779 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:51.978187 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:51.978190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:51.978192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:51.978658 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:51.978820 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:51.979431 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 02:49:51.979683 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":466,"thread_start_us":295,"threads_started":1}
I20260808 02:49:51.980743 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:51.962056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34831 (local address 127.0.0.1:42190)
0808 02:49:51.962848 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:51.962903 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:51.964847 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:51.965828 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:51.965834 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:51.966348 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:51.966664 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:51.966681 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:51.968600 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:51.968604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:51.969721 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:51.969734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:51.970867 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:51.970872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:51.971018 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:51.971634 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:51.971655 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:51.973766 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:51.976894 (+ 3128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:51.976907 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:51.976918 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:51.977501 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:51.977889 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:51.977893 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:51.977897 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:51.978098 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:51.978810 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:51.978817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:51.979036 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 02:49:51.979287 (+ 251us) client_negotiation.cc:240] Negotiation successful
0808 02:49:51.979858 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":124,"thread_start_us":319,"threads_started":1}
I20260808 02:49:51.983922 29815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:49:51.984907 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:51.984533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43396)
0808 02:49:51.984688 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:51.984802 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 02:49:51.985162 29748 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
I20260808 02:49:51.986608 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:51.986267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43408)
0808 02:49:51.986400 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:51.986467 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 02:49:51.986809 29815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:43149: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.25.137.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab.
Entry for principal kudu/127.25.137.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab.
I20260808 02:49:52.033728 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.253:43149
--webserver_interface=127.25.137.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab
--principal=kudu/127.25.137.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:34831,127.25.137.253:43149,127.25.137.252:37675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:49:52.036130 29815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } attempt: 1
I20260808 02:49:52.037374 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.036872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43418)
0808 02:49:52.037092 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.037216 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:49:52.038370 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.038004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43422)
0808 02:49:52.038154 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.038226 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 02:49:52.038559 29815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:43149: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
I20260808 02:49:52.106786 29815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } attempt: 2
I20260808 02:49:52.107972 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.107483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43424)
0808 02:49:52.107726 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.107828 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:49:52.109203 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.108742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43436)
0808 02:49:52.108916 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.108998 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260808 02:49:52.109414 29815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:43149: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
I20260808 02:49:52.200668 29815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } attempt: 3
I20260808 02:49:52.201933 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.201456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43450)
0808 02:49:52.201663 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.201758 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 02:49:52.203034 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.202675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43458)
0808 02:49:52.202820 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.202893 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:49:52.203228 29815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:43149: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
I20260808 02:49:52.346462 29815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } attempt: 4
I20260808 02:49:52.347734 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.347164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43466)
0808 02:49:52.347506 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.347599 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260808 02:49:52.348963 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.348569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43470)
0808 02:49:52.348739 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.348830 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260808 02:49:52.349248 29815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.253:43149: Network error: Client connection negotiation failed: client connection to 127.25.137.253:43149: connect: Connection refused (error 111)
W20260808 02:49:52.368152 29825 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:49:52.368464 29825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:52.368558 29825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:52.368667 29825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:49:52.376180 29825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:52.376262 29825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:52.376341 29825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:52.376415 29825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:52.376498 29825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:52.386258 29825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:34831,127.25.137.253:43149,127.25.137.252:37675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.253.keytab
--principal=kudu/127.25.137.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.253:43149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.137.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:52.387503 29825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:52.388756 29825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:52.400166 29834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:52.400410 29837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:52.400256 29831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:52.401361 29825 server_base.cc:1034] running on GCE node
I20260808 02:49:52.402209 29825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:52.403594 29825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:52.404862 29825 hybrid_clock.cc:648] HybridClock initialized: now 1786157392404795 us; error 33 us; skew 500 ppm
I20260808 02:49:52.405243 29825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:52 dist-test-slave-k5rr krb5kdc[29689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:52.408607 29825 init.cc:377] Logged in from keytab as kudu/127.25.137.253@KRBTEST.COM (short username kudu)
I20260808 02:49:52.410089 29825 webserver.cc:460] Webserver started at http://127.25.137.253:34809/ using document root <none> and password file <none>
I20260808 02:49:52.410765 29825 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:52.410868 29825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:52.411176 29825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:52.414350 29825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/data/instance:
uuid: "cd8f2bd01a8b4017a4001bf186dc334d"
format_stamp: "Formatted at 2026-08-08 02:49:52 on dist-test-slave-k5rr"
server_key: "196088f30f9bd1d055622af9aa8684fb"
server_key_iv: "86ffdcdcda0149a55ed4c31b75f4517c"
server_key_version: "encryptionkey@0"
I20260808 02:49:52.415120 29825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/wal/instance:
uuid: "cd8f2bd01a8b4017a4001bf186dc334d"
format_stamp: "Formatted at 2026-08-08 02:49:52 on dist-test-slave-k5rr"
server_key: "196088f30f9bd1d055622af9aa8684fb"
server_key_iv: "86ffdcdcda0149a55ed4c31b75f4517c"
server_key_version: "encryptionkey@0"
I20260808 02:49:52.420084 29825 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 02:49:52.423416 29846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:52.424608 29825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:49:52.424759 29825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/wal
uuid: "cd8f2bd01a8b4017a4001bf186dc334d"
format_stamp: "Formatted at 2026-08-08 02:49:52 on dist-test-slave-k5rr"
server_key: "196088f30f9bd1d055622af9aa8684fb"
server_key_iv: "86ffdcdcda0149a55ed4c31b75f4517c"
server_key_version: "encryptionkey@0"
I20260808 02:49:52.424948 29825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:52.437687 29825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:52.441797 29825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:52.442057 29825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:52.459465 29825 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.253:43149
I20260808 02:49:52.459472 29916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.253:43149 every 8 connection(s)
I20260808 02:49:52.460757 29825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/data/info.pb
I20260808 02:49:52.464387 29920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:52.466908 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 29825
I20260808 02:49:52.467092 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-1/wal/instance
I20260808 02:49:52.467640 26151 external_mini_cluster.cc:1428] Setting key 334aa2d925b1fbfa7f4800d380acaed1
I20260808 02:49:52.472647 29920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:49:52 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:52.494112 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.475621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42204 (local address 127.25.137.254:34831)
0808 02:49:52.475837 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:52.475844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:52.477040 (+ 1196us) server_negotiation.cc:407] Connection header received
0808 02:49:52.478980 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:52.478986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:52.479072 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:52.479184 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:52.480718 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.481658 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.483149 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.483726 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.483825 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:52.488112 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:52.488118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:52.488121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:52.488157 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:52.490865 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:52.491833 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:52.491837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:52.491839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:52.491967 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:52.492403 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:52.492406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:52.492408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:52.492686 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:52.492812 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:52.493680 (+ 868us) server_negotiation.cc:299] Negotiation successful
0808 02:49:52.493872 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:49:52.495589 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.476085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34831 (local address 127.0.0.1:42204)
0808 02:49:52.476852 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.476902 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:52.478700 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:52.479434 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:52.479443 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:52.479953 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:52.480516 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:52.480534 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.481830 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.481834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:52.482993 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:52.483006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.483955 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.483962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:52.484265 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:52.484983 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:52.485047 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:52.487849 (+ 2802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:52.491024 (+ 3175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:52.491035 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:52.491101 (+ 66us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:52.491710 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:52.492072 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:52.492076 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:52.492079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:52.492309 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:52.492817 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:52.492824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:52.493301 (+ 477us) client_negotiation.cc:769] Sending connection context
0808 02:49:52.493613 (+ 312us) client_negotiation.cc:240] Negotiation successful
0808 02:49:52.494150 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":321,"threads_started":1}
I20260808 02:49:52.497325 29920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.137.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.137.252@KRBTEST.COM" created.
Aug 08 02:49:52 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
I20260808 02:49:52.514426 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.498321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43482 (local address 127.25.137.253:43149)
0808 02:49:52.499123 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:52.499129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:52.499158 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:49:52.499204 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:52.499209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:52.499405 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:52.499525 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:52.500408 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.501552 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.503154 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.503723 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.503831 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:52.508889 (+ 5058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:52.508903 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:52.508916 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:52.508946 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:52.511668 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:52.512407 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:52.512413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:52.512416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:52.512527 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:52.512906 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:52.512910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:52.512912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:52.513316 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:52.513476 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:52.513877 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 02:49:52.514090 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":591,"thread_start_us":444,"threads_started":1}
I20260808 02:49:52.514431 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.497930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43482)
0808 02:49:52.498145 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.498170 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:52.498309 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:52.499574 (+ 1265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:52.499579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:52.500034 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:52.500215 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:52.500227 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.501698 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.501704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:52.502991 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:52.503007 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.503903 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.503909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:52.504091 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:52.504667 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:52.504684 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:52.508667 (+ 3983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:52.511818 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:52.511822 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:52.511826 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:52.512275 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:52.512626 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:52.512630 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:52.512632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:52.512800 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:52.513431 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:52.513440 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:52.513579 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 02:49:52.513719 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 02:49:52.513989 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 02:49:52.516981 29920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:49:52.518204 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.517650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38518)
0808 02:49:52.517860 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.518010 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 02:49:52.518468 29847 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
I20260808 02:49:52.520053 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.519650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38522)
0808 02:49:52.519819 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.519926 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:49:52.520364 29920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:37675: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.137.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab.
Entry for principal kudu/127.25.137.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab.
I20260808 02:49:52.547374 26151 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOXTGli/build/asan/bin/kudu
/tmp/dist-test-taskOXTGli/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.252:37675
--webserver_interface=127.25.137.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab
--principal=kudu/127.25.137.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.137.212:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.137.254:34831,127.25.137.253:43149,127.25.137.252:37675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:49:52.569708 29920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } attempt: 1
I20260808 02:49:52.571087 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.570532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38530)
0808 02:49:52.570789 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.570912 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 02:49:52.572255 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.571859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38534)
0808 02:49:52.572044 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.572130 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 02:49:52.572469 29920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:37675: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
I20260808 02:49:52.640676 29920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } attempt: 2
I20260808 02:49:52.641961 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.641399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38542)
0808 02:49:52.641635 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.641745 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260808 02:49:52.643039 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.642674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38552)
0808 02:49:52.642845 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.642919 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 02:49:52.643237 29920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:37675: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
I20260808 02:49:52.648473 29815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } attempt: 5
Aug 08 02:49:52 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157391, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
I20260808 02:49:52.662511 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.649135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43486)
0808 02:49:52.649324 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.649343 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:52.649482 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:52.649836 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:52.649842 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:52.650255 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:52.650493 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:52.650502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.651514 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.651519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:52.652396 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:52.652410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.653185 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.653191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:52.653418 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:52.654064 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:52.654083 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:52.656487 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:52.660158 (+ 3671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:52.660162 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:52.660165 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:52.660543 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:52.660938 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:52.660941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:52.660944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:52.661128 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:52.661729 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:52.661733 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:52.661903 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 02:49:52.662018 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 02:49:52.662262 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 02:49:52.662724 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.649282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43486 (local address 127.25.137.253:43149)
0808 02:49:52.649472 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:52.649478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:52.649500 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:49:52.649596 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:52.649601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:52.649685 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:52.649772 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:52.650647 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.651362 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.652548 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:52.653033 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:52.653153 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:52.656703 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:52.656710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:52.656715 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:52.656757 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:52.659997 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:52.660657 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:52.660663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:52.660666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:52.660820 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:52.661225 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:52.661230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:52.661233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:52.661612 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:52.661766 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:52.662154 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 02:49:52.662397 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:49:52.663676 29815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:49:52.664776 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.664326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38560)
0808 02:49:52.664520 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.664618 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 02:49:52.665838 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.665453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38576)
0808 02:49:52.665622 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.665707 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 02:49:52.666042 29815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:37675: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
I20260808 02:49:52.717288 29815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } attempt: 1
I20260808 02:49:52.718627 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.718043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38582)
0808 02:49:52.718271 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.718409 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 02:49:52.719735 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.719281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38588)
0808 02:49:52.719472 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.719567 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 02:49:52.719962 29815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:37675: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
I20260808 02:49:52.734460 29920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } attempt: 3
I20260808 02:49:52.735514 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.735043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38600)
0808 02:49:52.735256 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.735360 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:49:52.736819 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.736291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38614)
0808 02:49:52.736544 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.736637 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260808 02:49:52.737067 29920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:37675: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
I20260808 02:49:52.788216 29815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } attempt: 2
I20260808 02:49:52.789567 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.788985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38628)
0808 02:49:52.789236 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.789359 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 02:49:52.790659 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.790282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38636)
0808 02:49:52.790429 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.790527 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 02:49:52.790918 29815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:37675: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
I20260808 02:49:52.880324 29920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } attempt: 4
I20260808 02:49:52.881567 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.881061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38646)
0808 02:49:52.881297 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.881392 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 02:49:52.882668 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.882318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38652)
0808 02:49:52.882466 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.882545 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 02:49:52.882922 29920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:37675: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
W20260808 02:49:52.892961 29930 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:49:52.893329 29930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:49:52.893415 29930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:49:52.893529 29930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 02:49:52.897075 29815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } attempt: 3
I20260808 02:49:52.898138 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.897647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38666)
0808 02:49:52.897838 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.897951 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 02:49:52.899228 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.898819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38672)
0808 02:49:52.899011 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:52.899100 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260808 02:49:52.899436 29815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.137.252:37675: Network error: Client connection negotiation failed: client connection to 127.25.137.252:37675: connect: Connection refused (error 111)
W20260808 02:49:52.901510 29930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:49:52.901584 29930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:49:52.901638 29930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:49:52.901688 29930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:49:52.901748 29930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:49:52.911439 29930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.137.212:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.137.254:34831,127.25.137.253:43149,127.25.137.252:37675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/krb5kdc/kudu_127.25.137.252.keytab
--principal=kudu/127.25.137.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.137.252:37675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.137.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:49:52.912616 29930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:49:52.913980 29930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:49:52.925421 29940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:52.925446 29938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:49:52.925446 29943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:49:52.926035 29930 server_base.cc:1034] running on GCE node
I20260808 02:49:52.926761 29930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:49:52.928064 29930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:49:52.929322 29930 hybrid_clock.cc:648] HybridClock initialized: now 1786157392929228 us; error 58 us; skew 500 ppm
I20260808 02:49:52.929632 29930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:49:52 dist-test-slave-k5rr krb5kdc[29689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:49:52.932627 29930 init.cc:377] Logged in from keytab as kudu/127.25.137.252@KRBTEST.COM (short username kudu)
I20260808 02:49:52.933983 29930 webserver.cc:460] Webserver started at http://127.25.137.252:44921/ using document root <none> and password file <none>
I20260808 02:49:52.934634 29930 fs_manager.cc:359] Metadata directory not provided
I20260808 02:49:52.934733 29930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:49:52.935061 29930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:49:52.937991 29930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/data/instance:
uuid: "dd0afe5898634c74a5cab444369990fe"
format_stamp: "Formatted at 2026-08-08 02:49:52 on dist-test-slave-k5rr"
server_key: "3b2832f9ace8f8dfddbab03dda77fcd4"
server_key_iv: "b6e6225284421fe60114f0ee0e40a0f2"
server_key_version: "encryptionkey@0"
I20260808 02:49:52.938760 29930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/wal/instance:
uuid: "dd0afe5898634c74a5cab444369990fe"
format_stamp: "Formatted at 2026-08-08 02:49:52 on dist-test-slave-k5rr"
server_key: "3b2832f9ace8f8dfddbab03dda77fcd4"
server_key_iv: "b6e6225284421fe60114f0ee0e40a0f2"
server_key_version: "encryptionkey@0"
I20260808 02:49:52.943635 29930 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:49:52.947276 29953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:49:52.948588 29930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:49:52.948740 29930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/data,/tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/wal
uuid: "dd0afe5898634c74a5cab444369990fe"
format_stamp: "Formatted at 2026-08-08 02:49:52 on dist-test-slave-k5rr"
server_key: "3b2832f9ace8f8dfddbab03dda77fcd4"
server_key_iv: "b6e6225284421fe60114f0ee0e40a0f2"
server_key_version: "encryptionkey@0"
I20260808 02:49:52.948933 29930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:49:52.964284 29930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:49:52.968034 29930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:49:52.968284 29930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:49:52.985234 29930 rpc_server.cc:307] RPC server started. Bound to: 127.25.137.252:37675
I20260808 02:49:52.985252 30019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.137.252:37675 every 8 connection(s)
I20260808 02:49:52.986562 29930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/data/info.pb
I20260808 02:49:52.990309 30020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:49:52.992280 26151 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOXTGli/build/asan/bin/kudu as pid 29930
I20260808 02:49:52.992440 26151 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOXTGli/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157019169307-26151-0/minicluster-data/master-2/wal/instance
I20260808 02:49:52.992880 26151 external_mini_cluster.cc:1428] Setting key 110218d386c2d2f5f7909a17f05dd6fe
I20260808 02:49:53.001143 30020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:49:53 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
Aug 08 02:49:53 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:53.018090 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:52.994637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38674 (local address 127.25.137.252:37675)
0808 02:49:52.995501 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:52.995517 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:52.995576 (+ 59us) server_negotiation.cc:407] Connection header received
0808 02:49:52.997760 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:52.997788 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:52.998342 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:52.998805 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.000198 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.002291 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.003641 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.004505 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.004707 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.009791 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:53.009800 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:53.009812 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:53.009856 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:53.013493 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.014273 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.014283 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.014291 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.014465 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.014790 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.014793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.014795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.015529 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:53.015822 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:53.016280 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 02:49:53.016715 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":338,"threads_started":1}
I20260808 02:49:53.020090 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.004211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.254:34831 (local address 127.0.0.1:42216)
0808 02:49:53.005393 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:53.005450 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:53.005636 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:53.006044 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:53.006050 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:53.006686 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:53.006947 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:53.006961 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.008370 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.008376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:53.009596 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:53.009613 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.010362 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.010369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:53.010529 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.011098 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:53.011125 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:53.013881 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:53.017630 (+ 3749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:53.017639 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:53.017644 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:53.018076 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:53.018525 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:53.018529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:53.018532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:53.018692 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:53.019202 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:53.019208 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:53.019426 (+ 218us) client_negotiation.cc:769] Sending connection context
0808 02:49:53.019585 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 02:49:53.019808 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":964,"thread_start_us":839,"threads_started":1}
I20260808 02:49:53.020394 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.004204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.25.137.254:34831)
0808 02:49:53.004823 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.004830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.005552 (+ 722us) server_negotiation.cc:407] Connection header received
0808 02:49:53.005778 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.005782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.005849 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.005930 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.007128 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.008191 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.009737 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.010166 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.010256 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.014061 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:53.014066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:53.014069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:53.014103 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:53.017483 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.018217 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.018222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.018225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.018393 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.018837 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.018840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.018842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.019079 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:53.019197 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:53.019740 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 02:49:53.020020 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":313,"threads_started":1}
I20260808 02:49:53.021714 30020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:49:53 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
Aug 08 02:49:53 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.254@KRBTEST.COM
I20260808 02:49:53.035806 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.022312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.253:43149 (local address 127.0.0.1:43496)
0808 02:49:53.022477 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:53.022494 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:53.022583 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:53.022848 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:53.022853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:53.023278 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:53.023450 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:53.023463 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.024727 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.024732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:53.026095 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:53.026108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.027094 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.027100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:53.027297 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.027965 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:53.027987 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:53.030426 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:53.033633 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:53.033639 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:53.033643 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:53.033990 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:53.034332 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:53.034337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:53.034341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:53.034471 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:53.034944 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:53.034949 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:53.035085 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 02:49:53.035201 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 02:49:53.035468 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 02:49:53.035861 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.022433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.25.137.253:43149)
0808 02:49:53.022598 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.022601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.022618 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:49:53.022668 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.022671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.022727 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.022803 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.023597 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.024604 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.026250 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.026936 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.027056 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.030622 (+ 3566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:53.030630 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:53.030634 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:53.030676 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:53.033499 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.034097 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.034102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.034104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.034225 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.034565 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.034570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.034574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.034834 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:53.034960 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:53.035303 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 02:49:53.035496 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:49:53.036823 30020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:49:53.036986 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.021595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42228 (local address 127.25.137.254:34831)
0808 02:49:53.021748 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.021753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.021769 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:49:53.021881 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.021886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.021967 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.022046 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.023441 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.024453 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.026249 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.026929 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.027063 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.031588 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:53.031593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:53.031597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:53.031630 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:53.034296 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.034959 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.034965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.034969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.035102 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.035483 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.035489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.035491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.035778 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:53.036257 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:53.036401 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 02:49:53.036634 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 02:49:53 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.252@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
Aug 08 02:49:53 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.137.253@KRBTEST.COM
I20260808 02:49:53.050082 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.037502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38676)
0808 02:49:53.037667 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:53.037683 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:53.037781 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:53.038168 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:53.038172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:53.038554 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:53.038706 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:53.038717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.040023 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.040026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:53.041130 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:53.041145 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.042010 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.042018 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:53.042232 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.042758 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:53.042775 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:53.044641 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:53.047830 (+ 3189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:53.047833 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:53.047836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:53.048198 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:53.048576 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:53.048579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:53.048581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:53.048706 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:53.049328 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:53.049332 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:53.049446 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 02:49:53.049570 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 02:49:53.049828 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 02:49:53.050356 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.037747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38676 (local address 127.25.137.252:37675)
0808 02:49:53.037903 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.037907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.037921 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:49:53.037956 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.037961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.038028 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.038132 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.038881 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.039902 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.041264 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.041767 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.041915 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.044819 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:53.044823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:53.044827 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:53.044855 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:53.047664 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.048322 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.048328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.048331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.048476 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.048818 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.048823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.048827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.049199 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:53.049340 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:53.049705 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 02:49:53.049984 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":54}
I20260808 02:49:53.053164 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.038719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.25.137.253:43149)
0808 02:49:53.038875 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.038880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.038896 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:49:53.038994 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.038997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.039045 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.039110 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.040493 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.041523 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.043118 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.043519 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.043625 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.047580 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:53.047587 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:53.047592 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:53.047628 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:53.050620 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.051232 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.051237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.051240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.051464 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.051876 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.051881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.051883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.052198 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:53.052470 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:53.052624 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 02:49:53.052832 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:49:53.055883 30020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe: Bootstrap starting.
I20260808 02:49:53.057487 26151 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:49:53.059137 30020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:53.060124 30020 log.cc:824] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:53.059684 29778 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "60d20d409c8346f784c105f546c190b6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42228
I20260808 02:49:53.062861 30020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe: No bootstrap required, opened a new log
I20260808 02:49:53.069808 30020 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } }
I20260808 02:49:53.070135 30020 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd0afe5898634c74a5cab444369990fe, State: Initialized, Role: FOLLOWER
I20260808 02:49:53.070544 30020 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } }
I20260808 02:49:53.071408 30029 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } } }
I20260808 02:49:53.071851 30029 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:53.072384 30020 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:53.076625 29815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } attempt: 4
W20260808 02:49:53.081328 30044 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:49:53.081447 30044 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 02:49:53 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157391, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.254@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
I20260808 02:49:53.079624 29930 master_runner.cc:186] Error getting master registration for 127.25.137.254:34831: OK, instance_id { permanent_uuid: "60d20d409c8346f784c105f546c190b6" instance_seqno: 1786157391942024 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:49:53.091007 29930 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:49:53.092945 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.077302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38688)
0808 02:49:53.077498 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:53.077519 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:53.077650 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:53.078031 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:53.078037 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:53.078585 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:53.078867 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:53.078880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.080514 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.080519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:53.081809 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:53.081823 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.082895 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.082901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:53.083078 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.083646 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:53.083663 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:53.085978 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:53.090143 (+ 4165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:53.090149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:53.090153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:53.090661 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:53.091095 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:53.091101 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:53.091104 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:53.091262 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:53.091948 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:53.091953 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:53.092112 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 02:49:53.092267 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 02:49:53.092546 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 02:49:53.093044 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.077461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38688 (local address 127.25.137.252:37675)
0808 02:49:53.077652 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.077658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.077685 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:49:53.077770 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.077775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.077867 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.077976 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.079070 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.080274 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.081953 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.082749 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.082889 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.086196 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:53.086202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:53.086207 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:53.086249 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:53.089993 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.090806 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.090812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.090816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.090953 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.091386 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.091392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.091395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.091806 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:53.091979 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:53.092422 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 02:49:53.092677 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:49:53.099057 29815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6: Bootstrap starting.
I20260808 02:49:53.103255 29815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:53.104614 29815 log.cc:824] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:53.108094 29815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6: No bootstrap required, opened a new log
I20260808 02:49:53.115813 29815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } }
I20260808 02:49:53.116252 29815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d20d409c8346f784c105f546c190b6, State: Initialized, Role: FOLLOWER
I20260808 02:49:53.116789 29815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } }
I20260808 02:49:53.117831 30047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } } }
I20260808 02:49:53.118208 30047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:53.119154 29815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:53.119295 29778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 02:49:53.119542 29778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:53.122406 29778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } }
I20260808 02:49:53.124728 29989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60d20d409c8346f784c105f546c190b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dd0afe5898634c74a5cab444369990fe"
I20260808 02:49:53.125226 29989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:53.128647 29989 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60d20d409c8346f784c105f546c190b6 in term 1.
I20260808 02:49:53.129272 29743 leader_election.cc:302] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d20d409c8346f784c105f546c190b6, dd0afe5898634c74a5cab444369990fe; no voters:
I20260808 02:49:53.129778 30047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:49:53.137120 29778 leader_election.cc:288] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [CANDIDATE]: Term 1 election: Requested vote from peers cd8f2bd01a8b4017a4001bf186dc334d (127.25.137.253:43149), dd0afe5898634c74a5cab444369990fe (127.25.137.252:37675)
I20260808 02:49:53.137879 30047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [term 1 LEADER]: Becoming Leader. State: Replica: 60d20d409c8346f784c105f546c190b6, State: Running, Role: LEADER
I20260808 02:49:53.138453 30047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } }
I20260808 02:49:53.168455 30061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60d20d409c8346f784c105f546c190b6. Latest consensus state: current_term: 1 leader_uuid: "60d20d409c8346f784c105f546c190b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } } }
I20260808 02:49:53.168877 30061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:53.180495 30076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:49:53.182139 29920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } attempt: 5
W20260808 02:49:53.182814 29750 tablet.cc:2369] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:49:53.184870 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.160376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38704 (local address 127.25.137.252:37675)
0808 02:49:53.160569 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.160576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.164903 (+ 4327us) server_negotiation.cc:407] Connection header received
0808 02:49:53.165815 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.165821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.165916 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.166018 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.168332 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.169443 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.171672 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.172512 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.172618 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.176644 (+ 4026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:53.176650 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:53.176655 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:53.176698 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:53.180272 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.181292 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.181298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.181301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.181462 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.181982 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.181987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.181990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.182335 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:53.182559 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:53.184216 (+ 1657us) server_negotiation.cc:299] Negotiation successful
0808 02:49:53.184476 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 02:49:53 dist-test-slave-k5rr krb5kdc[29689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.137.253@KRBTEST.COM for kudu/127.25.137.252@KRBTEST.COM
I20260808 02:49:53.216394 30076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 02:49:53.220327 29965 tablet.cc:2369] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:49:53.222563 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.160348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42234 (local address 127.25.137.254:34831)
0808 02:49:53.161213 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.161219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.193445 (+ 32226us) server_negotiation.cc:407] Connection header received
0808 02:49:53.193718 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.193723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.193806 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.193874 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.197069 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.198067 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.199903 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.200497 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.200582 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.210491 (+ 9909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:53.210497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:53.210501 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:53.210547 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:53.217055 (+ 6508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.218070 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.218078 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.218082 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.218273 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.218779 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.218784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.218786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.219220 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:53.219519 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:53.221842 (+ 2323us) server_negotiation.cc:299] Negotiation successful
0808 02:49:53.222147 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:49:53.200799 29716 master_runner.cc:186] Error getting master registration for 127.25.137.253:43149: OK, instance_id { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" instance_seqno: 1786157392454533 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:49:53.223135 29716 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:49:53.230329 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.182746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.137.252:37675 (local address 127.0.0.1:38714)
0808 02:49:53.183133 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 02:49:53.183153 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:49:53.183278 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:49:53.185794 (+ 2516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:49:53.185800 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:49:53.186337 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:49:53.186554 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:53.186566 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.188147 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.188153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:53.189605 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:49:53.189622 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.190753 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.190771 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:49:53.191034 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.191703 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:49:53.191729 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:49:53.194556 (+ 2827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:49:53.220785 (+ 26229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:53.220791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:53.220795 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:53.221356 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:53.227959 (+ 6603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:49:53.227965 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:49:53.227969 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:49:53.228218 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:49:53.229081 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:49:53.229086 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:49:53.229254 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 02:49:53.229534 (+ 280us) client_negotiation.cc:240] Negotiation successful
0808 02:49:53.229922 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"spinlock_wait_cycles":21376}
I20260808 02:49:53.231354 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.183547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38714 (local address 127.25.137.252:37675)
0808 02:49:53.185446 (+ 1899us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.185452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.185470 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:49:53.185519 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.185524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.185597 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.186713 (+ 1116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.186856 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.187965 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.189814 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.190581 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.194791 (+ 4210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.195021 (+ 230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:53.195027 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:53.195031 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:53.195061 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:53.220132 (+ 25071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.221530 (+ 1398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.221535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.221538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.221697 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.228426 (+ 6729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.228432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.228435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.228866 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:53.230573 (+ 1707us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:53.230713 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 02:49:53.230983 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1795,"thread_start_us":9221,"threads_started":1}
I20260808 02:49:53.236799 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:53.194066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.25.137.253:43149)
0808 02:49:53.194269 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.194275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.212672 (+ 18397us) server_negotiation.cc:407] Connection header received
0808 02:49:53.214151 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.214159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.214254 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.214327 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.215931 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.217167 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.224709 (+ 7542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:53.225441 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:53.225543 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:53.229234 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:49:53.229241 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:49:53.229245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:49:53.229290 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:49:53.233169 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.234161 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.234167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.234171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.234376 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:49:53.234854 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:49:53.234860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:49:53.234864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:49:53.235237 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:49:53.235496 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:53.236162 (+ 666us) server_negotiation.cc:299] Negotiation successful
0808 02:49:53.236433 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:49:53.237814 29920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d: Bootstrap starting.
I20260808 02:49:53.241883 29920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d: Neither blocks nor log segments found. Creating new log.
I20260808 02:49:53.243206 29920 log.cc:824] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d: Log is configured to *not* fsync() on all Append() calls
I20260808 02:49:53.248456 29989 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [term 1 FOLLOWER]: Refusing update from remote peer 60d20d409c8346f784c105f546c190b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:49:53.249617 30061 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:49:53.252899 29920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d: No bootstrap required, opened a new log
I20260808 02:49:53.264451 29920 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } }
I20260808 02:49:53.264967 29920 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd8f2bd01a8b4017a4001bf186dc334d, State: Initialized, Role: FOLLOWER
I20260808 02:49:53.265584 29920 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } }
I20260808 02:49:53.274389 29920 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:49:53.274452 29884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60d20d409c8346f784c105f546c190b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cd8f2bd01a8b4017a4001bf186dc334d"
I20260808 02:49:53.275009 29884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:49:53.278738 29884 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60d20d409c8346f784c105f546c190b6 in term 1.
I20260808 02:49:53.279840 30087 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } } }
I20260808 02:49:53.280350 30087 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:53.280422 29883 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [term 1 FOLLOWER]: Refusing update from remote peer 60d20d409c8346f784c105f546c190b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:49:53.281442 30061 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:49:53.305540 30087 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60d20d409c8346f784c105f546c190b6. Latest consensus state: current_term: 1 leader_uuid: "60d20d409c8346f784c105f546c190b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } } }
I20260808 02:49:53.305958 30087 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:53.310887 30082 mvcc.cc:205] Tried to move back new op lower bound from 7316100682729656320 to 7316100682300301312. Current Snapshot: MvccSnapshot[applied={T|T < 7316100682729656320}]
I20260808 02:49:53.319994 30029 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60d20d409c8346f784c105f546c190b6. Latest consensus state: current_term: 1 leader_uuid: "60d20d409c8346f784c105f546c190b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } } }
I20260808 02:49:53.320411 30029 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:53.325093 30061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "60d20d409c8346f784c105f546c190b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } } }
I20260808 02:49:53.325358 30061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:53.325763 30061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "60d20d409c8346f784c105f546c190b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } } }
I20260808 02:49:53.325979 30061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6 [sys.catalog]: This master's current role is: LEADER
I20260808 02:49:53.326869 30076 catalog_manager.cc:1347] Generated new cluster ID: b7373e8a072445aa8053d083fb9c8f4d
I20260808 02:49:53.326977 30076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:49:53.333022 30086 mvcc.cc:205] Tried to move back new op lower bound from 7316100682729656320 to 7316100682300301312. Current Snapshot: MvccSnapshot[applied={T|T < 7316100682729656320}]
I20260808 02:49:53.336429 30096 mvcc.cc:205] Tried to move back new op lower bound from 7316100682729656320 to 7316100682300301312. Current Snapshot: MvccSnapshot[applied={T|T < 7316100682729656320}]
I20260808 02:49:53.343514 30029 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60d20d409c8346f784c105f546c190b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } } }
I20260808 02:49:53.343828 30029 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:53.353312 30087 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60d20d409c8346f784c105f546c190b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d20d409c8346f784c105f546c190b6" member_type: VOTER last_known_addr { host: "127.25.137.254" port: 34831 } } peers { permanent_uuid: "cd8f2bd01a8b4017a4001bf186dc334d" member_type: VOTER last_known_addr { host: "127.25.137.253" port: 43149 } } peers { permanent_uuid: "dd0afe5898634c74a5cab444369990fe" member_type: VOTER last_known_addr { host: "127.25.137.252" port: 37675 } } }
I20260808 02:49:53.353613 30087 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:49:53.370654 30076 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:49:53.372723 30076 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:49:53.388422 30114 catalog_manager.cc:1259] Loaded cluster ID: b7373e8a072445aa8053d083fb9c8f4d
I20260808 02:49:53.388556 30114 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d: loading cluster ID for follower catalog manager: success
I20260808 02:49:53.392458 30114 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d: acquiring CA information for follower catalog manager: success
W20260808 02:49:53.393857 30114 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260808 02:49:53.419891 30076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60d20d409c8346f784c105f546c190b6: Generated new TSK 0
I20260808 02:49:53.421173 30076 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:49:53.571014 30023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.557189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.25.137.252:37675)
0808 02:49:53.557388 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.557395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.568742 (+ 11347us) server_negotiation.cc:407] Connection header received
0808 02:49:53.569323 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.569330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.569421 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.569509 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.570736 (+ 1227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38724: BlockingRecv error: recv got EOF from 127.0.0.1:38724 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 02:49:53.574410 30024 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.565258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.25.137.254:34831)
0808 02:49:53.565512 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.565520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.571908 (+ 6388us) server_negotiation.cc:407] Connection header received
0808 02:49:53.572210 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.572220 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.572317 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.572394 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.574116 (+ 1722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42238: BlockingRecv error: recv got EOF from 127.0.0.1:42238 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 02:49:53.577939 29925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.575520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43528 (local address 127.25.137.253:43149)
0808 02:49:53.575733 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.575740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.576308 (+ 568us) server_negotiation.cc:407] Connection header received
0808 02:49:53.576563 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.576569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.576670 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.576749 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.577707 (+ 958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43528: BlockingRecv error: recv got EOF from 127.0.0.1:43528 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 02:49:53.620594 29925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.611300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.25.137.253:43149)
0808 02:49:53.611508 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.611514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.618358 (+ 6844us) server_negotiation.cc:407] Connection header received
0808 02:49:53.618955 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.618962 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.619099 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.619178 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.620285 (+ 1107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43540: BlockingRecv error: recv got EOF from 127.0.0.1:43540 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 02:49:53.663483 29925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.654390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43554 (local address 127.25.137.253:43149)
0808 02:49:53.654629 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.654635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.661533 (+ 6898us) server_negotiation.cc:407] Connection header received
0808 02:49:53.662023 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.662030 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.662122 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.662193 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.663258 (+ 1065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43554: BlockingRecv error: recv got EOF from 127.0.0.1:43554 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 02:49:53.671226 29925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.669046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43558 (local address 127.25.137.253:43149)
0808 02:49:53.669240 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.669264 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.669793 (+ 529us) server_negotiation.cc:407] Connection header received
0808 02:49:53.669962 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.669971 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.670060 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.670145 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.670980 (+ 835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43558: BlockingRecv error: recv got EOF from 127.0.0.1:43558 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 02:49:53.679817 29925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.677799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43568 (local address 127.25.137.253:43149)
0808 02:49:53.677986 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.677993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.678467 (+ 474us) server_negotiation.cc:407] Connection header received
0808 02:49:53.678663 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.678667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.678756 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.678849 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.679585 (+ 736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43568: BlockingRecv error: recv got EOF from 127.0.0.1:43568 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 02:49:53.693044 29925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.690868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.25.137.253:43149)
0808 02:49:53.691059 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.691066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.691604 (+ 538us) server_negotiation.cc:407] Connection header received
0808 02:49:53.691813 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.691817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.691891 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.691977 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.692721 (+ 744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43570: BlockingRecv error: recv got EOF from 127.0.0.1:43570 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 02:49:53.714620 29925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.712375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.25.137.253:43149)
0808 02:49:53.712589 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.712597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.713115 (+ 518us) server_negotiation.cc:407] Connection header received
0808 02:49:53.713348 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.713352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.713432 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.713530 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.714358 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43586: BlockingRecv error: recv got EOF from 127.0.0.1:43586 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 02:49:53.751971 29925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.749721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43602 (local address 127.25.137.253:43149)
0808 02:49:53.749933 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.749940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.750474 (+ 534us) server_negotiation.cc:407] Connection header received
0808 02:49:53.750696 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.750701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.750775 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.750880 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.751685 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43602: BlockingRecv error: recv got EOF from 127.0.0.1:43602 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 02:49:53.823742 29925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.821198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43604 (local address 127.25.137.253:43149)
0808 02:49:53.821448 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.821455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.822017 (+ 562us) server_negotiation.cc:407] Connection header received
0808 02:49:53.822254 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.822260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.822379 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.822507 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.823440 (+ 933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43604: BlockingRecv error: recv got EOF from 127.0.0.1:43604 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260808 02:49:53.957450 29925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:53.955516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43614 (local address 127.25.137.253:43149)
0808 02:49:53.955719 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:53.955725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:53.956088 (+ 363us) server_negotiation.cc:407] Connection header received
0808 02:49:53.956260 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:53.956267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:53.956357 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:53.956457 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:53.957203 (+ 746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43614: BlockingRecv error: recv got EOF from 127.0.0.1:43614 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:49:54.083446 30044 catalog_manager.cc:1259] Loaded cluster ID: b7373e8a072445aa8053d083fb9c8f4d
I20260808 02:49:54.083556 30044 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe: loading cluster ID for follower catalog manager: success
I20260808 02:49:54.086108 30044 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe: acquiring CA information for follower catalog manager: success
I20260808 02:49:54.087960 30044 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dd0afe5898634c74a5cab444369990fe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 02:49:54.219004 29925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:49:54.217311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43628 (local address 127.25.137.253:43149)
0808 02:49:54.217527 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:54.217533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:54.217686 (+ 153us) server_negotiation.cc:407] Connection header received
0808 02:49:54.217897 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:54.217901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:54.217975 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:54.218064 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:49:54.218813 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43628: BlockingRecv error: recv got EOF from 127.0.0.1:43628 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:49:54.396886 30114 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd8f2bd01a8b4017a4001bf186dc334d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:49:54.740362 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:54.734545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43642 (local address 127.25.137.253:43149)
0808 02:49:54.735072 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:54.735079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:54.735143 (+ 64us) server_negotiation.cc:407] Connection header received
0808 02:49:54.735387 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:54.735391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:54.735546 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:54.735656 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:54.736114 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:54.736826 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:54.738177 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:54.738601 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:54.738715 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:54.739230 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:54.739752 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:54.739899 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:54.739994 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 02:49:54.740079 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":263,"threads_started":1}
I20260808 02:49:54.751865 30214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:54.746602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38730 (local address 127.25.137.252:37675)
0808 02:49:54.747167 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:54.747171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:54.747187 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:49:54.747276 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:54.747280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:54.747437 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:54.747574 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:54.747878 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:54.748560 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:54.749915 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:54.750313 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:54.750400 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:54.750800 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:54.751316 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:54.751422 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:54.751507 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 02:49:54.751571 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":372,"threads_started":1}
I20260808 02:49:54.764822 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:54.757266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.25.137.254:34831)
0808 02:49:54.757996 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:54.758002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:54.758554 (+ 552us) server_negotiation.cc:407] Connection header received
0808 02:49:54.758823 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:54.758829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:54.759036 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:54.759215 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:54.759990 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:54.761115 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:54.762860 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:54.763250 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:54.763361 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:54.763833 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:54.764166 (+ 333us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:54.764268 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:54.764427 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 02:49:54.764513 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":385,"threads_started":1}
I20260808 02:49:54.765815 30214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:54.757395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38742 (local address 127.25.137.252:37675)
0808 02:49:54.757562 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:54.757568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:54.758813 (+ 1245us) server_negotiation.cc:407] Connection header received
0808 02:49:54.759067 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:54.759072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:54.759148 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:54.759212 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:54.760106 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:54.761138 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:54.763449 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:54.764032 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:54.764124 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:54.764817 (+ 693us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:54.764932 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:54.765044 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:54.765477 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 02:49:54.765550 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:49:54.766353 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:49:54.758919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43654 (local address 127.25.137.253:43149)
0808 02:49:54.759111 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 02:49:54.759116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:49:54.759684 (+ 568us) server_negotiation.cc:407] Connection header received
0808 02:49:54.759869 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:49:54.759874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:49:54.759961 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:49:54.760039 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:49:54.760522 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:54.761291 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:54.763449 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:49:54.764053 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:49:54.764134 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:49:54.765138 (+ 1004us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:49:54.765231 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:49:54.765306 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:49:54.766026 (+ 720us) server_negotiation.cc:299] Negotiation successful
0808 02:49:54.766106 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:49:54.770133 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29716
I20260808 02:49:54.771075 29809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:54.881356 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 29716
I20260808 02:49:54.905061 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29825
I20260808 02:49:54.906225 29911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:55.017112 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 29825
I20260808 02:49:55.039057 26151 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOXTGli/build/asan/bin/kudu pid 29930
I20260808 02:49:55.040139 30013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:49:55.147472 26151 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOXTGli/build/asan/bin/kudu with pid 29930
2026-08-08T02:49:55Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5248 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10070 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (375816 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST